NInfer Windows DSpine
accelerating Qwen3.8-27B on RTX 5090
2 min readupdated
A bundle combining a Windows port of the NInfer inference engine (CUDA 13.1) and Tencent's DSpine draft model for speculative decoding (an acceleration technique where a small model predicts a batch of tokens at once and the main model verifies them in a single pass). Built specifically for a single RTX 5090 (32 GB, sm_120a architecture), it pairs with the base Qwen3.8-27B or the Coder390 fine-tune. Unlike standard inference, DSpine proposes 15 tokens per step without degrading final output quality. Ideal for owners of flagship NVIDIA GPUs running local LLMs on Windows who want maximum throughput for coding and text generation.
Key features
- Ready-to-use Windows engine: includes a precompiled
ninfer-serve-dspine.exewith an OpenAI-compatible API on port 8080. - NVFP4 format: quantizing the draft model's tables shrinks its footprint from 2218.7 MiB to 1347.3 MiB, saving around 869 MiB of VRAM with no speed penalty.
- Lossless accuracy: the target model verifies 15 draft tokens per step, preserving identical greedy output (at temperature 0).
- Two adapter variants: separate
officialsuffixes for standard Qwen3.8-27B andcoder390for the specialized coding fine-tune.
What you can do
- Speed up your local AI coding assistant: hook up the OpenAI-compatible
/v1/chat/completionsendpoint to your IDE or scripts using--spec dspine --draft-tokens 15. - Build a hybrid model file: merge the base Qwen3.8-27B DFlash2 checkpoint with a prebuilt NVFP4 suffix via simple byte-level concatenation without requantizing.
Example
python scripts\fuse_dspine_draft.py models\official-dflash2.ninfer draft\dspine-draft-official-nvfp4tab.ninfer models\official.dspine.ninfer
launch\serve-dspine-official.batRead more
DetailsThroughput with NVFP4 tables (RTX 5090): 339.3 tok/s on code (37.1% token acceptance), 211.3 tok/s on prose, and 858.4 tok/s on counting.
Memory: uses around 24.36 GiB VRAM after loading, leaving 8.59 GiB free.
Limitations: the Qwen3.8-27B and Coder390 weights are not included in the repo and must be downloaded separately in DFlash2 format; suffixes are tied to specific models and are not interchangeable.
License: Apache 2.0.










Comments