ot-distill-qwen3-8b-t9-6ep-lora
GLM 5.3 Flash agentic reasoning for Qwen3-8B
2 min readupdated
A set of LoRA adapters (lightweight add-ons for a base model) that trains the compact Qwen/Qwen3-8B LLM to tackle complex agentic tasks using step-by-step reasoning. The model was distilled (trained to mimic a larger model) on 2,701 Terminus-2 agent trajectories from GLM 5.3 Flash with high reasoning effort, preserving all reasoning blocks in the training data. Unlike standard Qwen3-8B, this adapter is tailored for multi-step agentic behavior using the OpenThoughts-Agent-SFT-3.16K dataset and requires a higher generation temperature to prevent looping. It is a solid fit for local AI agent developers who need a lightweight 8B model with planning capabilities.
Key features
- Chain-of-thought distillation: Trained on 2,701 examples from the
T9-3.16k-sft-v0configuration while preserving fulltags on every assistant turn. - High adapter rank: Uses LoRA rank 128 (alpha 256) across all key attention and MLP linear layers (
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj). - Long context training: Sequences were rendered with Qwen3 tokens and truncated to 32,768 tokens (65.4M tokens per epoch).
- Per-epoch checkpoints: The repository organizes weights into separate folders for different training stages (6 epochs, 1,014 steps).
What you can do
- Spin up a local AI agent server: Attach the adapter to the base
Qwen/Qwen3-8Bmodel viavllmwith the reasoning parser enabled using--reasoning-parser qwen3. - Merge weights into the base model: Fuse the LoRA with the original Qwen3-8B using the
peftlibrary (merge_and_unload()) for downstream quantization or deployment. - Compare quality across epochs: Test different training stages from the repository or benchmark the 8B version against its larger sibling,
ot-distill-qwen38-27b-t9-6ep-lora.
Example
vllm serve Qwen/Qwen3-8B --enable-lora --max-lora-rank 128 --reasoning-parser qwen3 \
--lora-modules t9-8b-e6=Read more
DetailsTraining: 6 epochs (1,014 steps), effective batch size 16 (4 GPUs × 4 gradient accumulation), bf16 precision, peak lr 3e-4 with cosine decay.
Training metrics: Average training loss reached 1.366 / 1.190 (partial epoch data).
Important caveat: Set
temperature = 1.0 during generation—early versions got stuck in repetition loops at Qwen3's default temperature of 0.6.Safety: The author plans to add misalignment evaluation results (StrongREJECT, XSTest, CoCoNot, MASK) judged by a local Llama-3.3-70B instance.










Comments