Qwen3.8-27B-Decision-v0.4: a model for choice probability scoring and decision-making
2 min readupdated
Built on Qwen3.8-27B (a 26.9B parameter model), Qwen3.8-27B-Decision-v0.4 is trained for decision-making tasks. It takes a state, a question, and 2 to 255 ordered response options, then scores their probabilities in a single forward pass. This is an independent implementation of a Jev-style decision interface, built for developers and researchers automating option selection in complex conversations or business workflows.
Key features
- Architecture: hybrid full-attention and linear-attention (recurrent) layers paired with a trained decision head.
- Training: fine-tuned using rank-32 LoRA (a lightweight adapter on top of the base model) and soft-target cross-entropy across 1.2M records.
- Inference: a dedicated scorer uses an exact tokenizer template to evaluate only the final decision position.
- Performance: beats Jev 1.14 reference baselines across several benchmarks (such as 65.17 vs. 60.70 on Decision Index Public).
What you can do
- Support automation: plug it into customer support systems to score an operator's or agent's next steps against company guidelines.
- Option analysis: rank candidate responses or actions in conversational systems.
- Benchmark testing: run evaluations on complex, structured tasks with multiple alternative choices.
Example
cd decision-model
python predict.py --state "The customer would like to reschedule a flight. The booking number is missing." \
--question "Which action should the agent take next?" \
--option "Ask for the booking number" --option "Confirm the flight has been changed"Read more
Details• Benchmarks: Decision Index Public — 65.17, JevBench Public — 88.74%, DecisionBench — 77.04%.
• Hardware & VRAM: 12 shards totaling 56.34 GB; a 64K-token pass on an H100 80 GB hit 65.46 GiB peak memory.
• Limitations: inputs exceeding the context limit throw an error instead of truncating options; accuracy on ordinal scales trails the reference baseline.
• Base model: Qwen/Qwen3.8-27B under the Apache 2.0 license.










Comments