Malaysian-Qwen2.5-7B-Instruct
adapting the language model for Malaysian context and dialects
1 min readupdated
This model is a further fine-tune of the base Qwen2.5-7B-Instruct LLM (large language model) trained on a 1.5 billion token dataset. It uses LoRA (a small add-on adapter for the base model) and supports responses in Mandarin, Tamil, Jawi, Manglish, and various regional dialects across Malaysia. The model excels at multi-step tasks involving local legal, political, and religious material, making it a great tool for developers and researchers who need high-quality text processing tailored to the region.
Key features
- Support for multiple regional Malaysian dialects and languages (including Jawi and Manglish).
- Code writing in local languages and dialects.
- Handling multi-step context, including Malaysian legislation and culture.
What you can do
- Integrate it into chatbots and support services for users in Malaysia.
- Use it to generate and analyze regional legal or political documentation.
Example
Using the model to generate responses and work with Malaysian context via standard Hugging Face libraries with attached LoRA weights.
Read more
DetailsBenchmarks: average MalayMMLU (0-shot) accuracy reaches 69.26% for the Rank 128 revision.
Training: LoRA on q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, embed_tokens, and lm_head projections with an 8192 context length.
Hardware: an 8x H100 node (supported by SNS and Nvidia).










Comments