Onyx-2B
a compact reasoning LoRA adapter built on Qwen3.5-2B
2 min read
Onyx-2B is a fine-tune of the Qwen/Qwen3.5-2B language model by AjelixAI, released as a LoRA (a lightweight adapter layered on top of the base model). Trained on a proprietary internal instruction dataset for logic and math tasks, it retains the base model's compact 2.24B-parameter footprint. Compared to the original Qwen3.5-2B, it delivers higher accuracy without bloating response length — its reasoning traces use virtually the same number of tokens. This adapter is a solid fit for researchers and developers who need a small local LLM for experimenting with step-by-step reasoning (thinking mode).
Key features
- Higher accuracy without extra verbosity: On a 16-task evaluation set, accuracy jumped from 37.5% to 56.3% while keeping the same average output length (842 vs. 843 tokens).
- Minimal trainable parameters: The LoRA adapter (r=32, α=32) modifies only 21.8M parameters — just 0.98% of the base model.
- Thinking mode support: Works with the standard chat template when
enable_thinking=Trueis set. - Permissive license: Released under the Apache-2.0 license, inherited from the base model.
What you can do
- Test local reasoning: Run the model on low-end hardware or in Colab to tackle arithmetic, algebra, combinatorics, and basic computer science problems.
- Plug into research pipelines: Load it via
transformersandpeftto benchmark how compact models perform in thinking mode.
Example
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
base = "Qwen/Qwen3.5-2B"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(
AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="auto"), "AjelixAI/Onyx-2B"
)
Read more
DetailsEvaluation: 9/16 (56.3%) vs. 6/16 (37.5%) for Qwen3.5-2B on a private set of 16 short tasks (greedy decoding, up to 1024 tokens).
Training: Unsloth + TRL, bfloat16 precision, 4096-token max sequence length, proprietary dataset.
Limitations: The evaluation sample is very small, and standard benchmarks (MMLU, GSM8K, HumanEval) were not run; the model inherits the typical weaknesses of 2.2B base models (limited world knowledge and a risk of hallucinations).










Comments