Qwen3.5-4B 4k GRPO maturebass
a math reasoning model
1 min readupdated
This model is a specialized fine-tune of the Qwen/Qwen3.5-4B base LLM (large language model), trained using GRPO (group relative policy optimization, a reinforcement learning method). It is built to solve math problems within a strict 4,096-token response limit. Trained on the DeepScaleR dataset to sharpen logical, step-by-step deduction, it is a solid fit for researchers and developers looking for a compact model with stronger math reasoning.
Key features
- Base model: Qwen/Qwen3.5-4B, roughly 4.5B parameters.
- Training algorithm: GRPO with token-level loss and group reward normalization.
- Generation limit: fixed token budget with max_new_tokens set to 4096.
- Correctness reward: binary scoring based on the final answer inside \boxed{} tags.
What you can do
- Run it in services or local scripts via the transformers library to tackle complex math problems.
- Spin up inference with vLLM for fast benchmarking and testing.
- Study how the model handles step-by-step deduction on competition-level or engineering calculations.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "budget-internalization-iclr2027/qwen3.5-4b-4k-grpo-multiseed-seed42-maturebass-s300"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
Read more
DetailsAlgorithm: GRPO (leave-one-out baseline, group reward normalization)
Data: DeepScaleR (math), up to 3 epochs
Optimizer: Adam, cosine LR, peak LR 5e-7
Checkpoint: step 300, seed 42, BF16 weights










Comments