qwen2.5-0.5b-coder-instruct-oss
Code generation companion
2 min readupdated
This is a lightweight LoRA (a small add-on on top of a base model) built on the compact Qwen/Qwen2.5-0.5B-Instruct model and trained on the OSS-Instruct dataset for programming tasks. The model specializes in writing code in Python, C++, Rust, and TypeScript, delivering high speed with minimal hardware requirements. It works great for local integration into development environments, automating routine tasks, and rapidly generating algorithms. The dataset is decontaminated against data leaks, and the model is released under the permissive Apache 2.0 license.
Key features
- Significant accuracy boost: Shows improved metrics over the base version on unlabeled test splits.
- Performance: Lightweight architecture ensures low latency and makes it suitable for edge device deployment.
- Apache 2.0 license: Free for both commercial and academic use.
- Compatibility: Integrates seamlessly with standard
transformersandpeftlibraries.
What you can do
- IDE integration: Set up a local assistant to write code snippets without relying on third-party APIs.
- Algorithm generation: Quickly generate functions in Python, Rust, or C++ from plain text descriptions.
- Test automation: Generate unit tests and verify code correctness.
Example
Inference example using the base model and the PEFT adapter:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_id = "vennyvirtuoso/qwen2.5-0.5b-coder-instruct-oss"
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
problem = "Write an efficient Rust function..."
messages = [{"role": "user", "content": problem}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
code = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(code)
Read more
DetailsPre-trained test perplexity: 2.92
Post-fine-tuning perplexity: 1.83 (37.2% relative reduction)
Test set size: 1,500 algorithmic problems
Training epochs: 2.0 (3,126 optimization steps)










Comments