ganit-1.5b-grpo-v2
LoRA adapter for step-by-step math problem solving
2 min readupdated
Created by Mayank Parashar, this LoRA (a lightweight add-on for a base model) targets the compact Qwen/Qwen2.5-1.5B-Instruct language model. It trains the model to solve grade-school math problems using step-by-step reasoning and output the final answer in a strict format using and tags. Trained via GRPO (a reinforcement learning algorithm based on group-relative response scoring) on the GSM8K dataset with verifiable rewards, the adapter boosts base model accuracy on the test split from 44.0% to 74.7% while cutting down average response length. It is a solid fit for developers who need compact, format-predictable models for English math reasoning.
Key features
- Higher accuracy on GSM8K: Accuracy jumped from 44.0% on the base model to 74.7% across a 300-problem test set.
- Strict output formatting: Compliance with the
andtag structure rose from 86.7% to 99.0%. - More concise reasoning: Average generation length dropped from 723 to 634 characters alongside the quality gains.
- Self-consistency support: Generating 4 responses pushes majority-vote accuracy to 76.0% (pass@4 reaches 87.0%).
What you can do
- Power educational tools: Integrate it into English-language tutoring apps and practice platforms for automated grading and step-by-step walkthroughs of school math problems.
- Automate answer parsing: Drop it into pipelines where you need to reliably extract the final number from the
tag without heavy post-processing. - Use as a GRPO reference: Reuse the training setup (TRL
GRPOTrainer, LoRA r=16) as a baseline for your own RL fine-tuning experiments on compact LLMs.
Example
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(model, "Mahir28/ganit-1.5b-grpo-v2")
Read more
DetailsAccuracy by problem difficulty: easy — 85.7% (N=112), medium — 68.1% (N=138), hard — 68.0% (N=50); pass@1 = 68.7%.
LoRA parameters: r = 16, α = 32, dropout = 0.05 (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj layers).
Training: 750 steps via TRL GRPOTrainer, 8 generations per prompt, KL β = 0.04, lr = 2e-05 (cosine), batch size 8 × 4.
Reward function: correctness, format, and tag_count with equal weights [1.0, 1.0, 1.0].










Comments