Llama-3.2-3B-Instruct-gsm8k-aug-nl-explicit-cot
a math problem-solving model
2 min readupdated
This model is a fine-tuned version of Llama-3.2-3B-Instruct, optimized to generate natural Chain-of-Thought reasoning and solve grade-school math problems. Trained on the zen-E/GSM8k-Aug-NL dataset as part of research into implicit reasoning, it was built using LoRA (a lightweight adapter on top of the base model) and then fully merged into the main weights. It is a solid fit for developers and researchers looking for a compact LLM with reliable math capabilities.
Key features
- Built on the efficient 3.2-billion-parameter Llama-3.2-3B-Instruct architecture.
- Tailored for step-by-step word problem solving with explicit intermediate reasoning steps.
- Hits 71.34% accuracy on the GSM8K test set and 75.40% on the SVAMP dataset.
- Ships as a standard checkpoint in bf16 (bfloat16) precision with the default vocabulary.
What you can do
- Integrate the model into local math assistants or educational bots for checking homework.
- Benchmark and test reasoning methods in compact LLMs.
- Plug it into text-processing pipelines that require detailed step-by-step problem breakdowns.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "hanseungwook/Llama-3.2-3B-Instruct-gsm8k-aug-nl-explicit-cot"
tokenizer = AutoTokenizer.from_pretrained(name, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(name, torch_dtype=torch.bfloat16, device_map="auto")
question = "Janet's ducks lay 16 eggs per day..."
inputs = tokenizer([question], return_tensors="pt").to(model.device)
output = model.generate(**inputs, do_sample=False, max_new_tokens=256)
print(tokenizer.decode(output[0, inputs["input_ids"].size(1):], skip_special_tokens=True))
Read more
Details• Benchmark accuracy: GSM8K test — 71.34%, SVAMP — 75.40%, MultiArith — 100.00%.
• Training: LoRA (r=128, alpha=32) on 4 GPUs using the AdamW optimizer.
• Requirements: compatible with standard transformers and PyTorch libraries in bf16 mode.
• License: distributed under the Llama 3.2 Community License.










Comments