HDM-Qwen3.8-27B
smart reasoning router for Qwen
2 min readupdated
A compact routing module (1.31M parameters, weighing just 5.3 MB) accompanying the paper "The Value of Thinking: Learning When to Reason in Hybrid Decision Models." It attaches to a frozen Qwen/Qwen3.8-27B base model to stop it from wasting extra tokens on long chains of thought when a direct answer works just fine. First, the main model runs a single quick pass without generating text, while the router evaluates the hidden state and eight response statistics to predict whether reasoning will actually improve the final output. Built for developers and researchers deploying large LLMs who want fine-grained control over the trade-off between accuracy and compute.
Key features
- Evaluates four possible outcomes: whether reasoning will fix an error from a quick answer or just make things worse.
- Controlled via a cost threshold
λ, which can be set per request without retraining the router. - Virtually zero memory overhead — the weights file takes up only 5.3 MB, and the base Qwen3.8-27B model remains untouched.
- Runs on the SGLang 0.5.9 backend (
transformersis also supported, though feature differences can alter routing behavior).
What you can do
- Hook it up to a Qwen3.8-27B inference server via the
hde-serveutility to cut down on reasoning tokens for straightforward multiple-choice questions. - Set up dynamic request routing in your API by passing the
lambdaparameter inside theroutingfield to tune thinking depth per task. - Retrain the router for your own base model, a different layer, or another compute precision — according to the authors, fitting takes just a few minutes.
Example
from huggingface_hub import hf_hub_download
from hde.core.block import DecisionHead
path = hf_hub_download("fredreick/HDM-Qwen3.8-27B", "head-27b-mmlu-pro.safetensors")
block = DecisionHead.load(path)
Read more
DetailsMMLU-Pro benchmark (346 held-out questions across chemistry, economics, and psychology): at λ = 0.05, accuracy hits 89.0% (compared to 90.2% with reasoning always on and 65.0% with it off), using 78% of the reasoning tokens.
JevBench benchmark (231 questions, zero fine-tuning on this dataset): at λ = 0, accuracy reaches 98.7% versus 99.1% in always-think mode, using 82% of the tokens.
Training: 1,154 multiple-choice questions from MMLU-Pro with a 4,096-token reasoning budget.
Limitations: tied to a specific
Qwen/Qwen3.8-27B revision, layer -1, and coded readout; non-multiple-choice question formats haven't been tested, and the router still frequently sends hopeless questions into reasoning.









Comments