Qwen3.8-27B-DFlash2
a draft model for accelerating Qwen3.8-27B generation
2 min readupdated
DFlash 2 is an auxiliary draft model for speculative decoding (a method for speeding up token generation), built specifically for the Qwen/Qwen3.8-27B language model. Rather than running as a standalone chatbot, it operates inside an inference server, predicting entire blocks of tokens in a single pass for the base model to verify. Powered by block diffusion and two-step dynamic convolutions, this approach guarantees completely lossless output: both greedy decoding and sampling distributions match the original model identically. It is ideal for production deployments that demand low-latency responses.
Key features
- Up to 3.43× faster generation compared to standard autoregressive decoding on the same tasks.
- Block-diffusion drafter that predicts an entire block of tokens in a single forward pass.
- Completely lossless: matches the target model's output distribution and results identically.
- Longer token acceptance lengths compared to standard methods like MTP and DSpark.
What you can do
- Spin up an inference server with SGLang, attaching the model as a speculative drafter for Qwen/Qwen3.8-27B.
- Configure a vLLM deployment using speculative decoding settings to speed up request throughput.
- Benchmark performance on coding and math tasks (HumanEval, MATH-500, GSM8K).
Example
Launching the server with SGLang:
python -m sglang.launch_server \
--model-path Qwen/Qwen3.8-27B \
--speculative-algorithm DFLASH \
--speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
--speculative-num-draft-tokens 8Read more
DetailsBenchmarks (on a single NVIDIA H200): speedup reaches 3.43× on GSM8K (236.1 tok/s vs. 68.9 tok/s at concurrency 1) and 3.34× on MATH-500.
Speculation block size: 8 tokens (7 draft tokens per verification step).
Compatibility: SGLang and vLLM inference engines.










Comments