Qwen3.8-Flash-Next-WV-v5.1-Sub2.5bpw-SM120
ultra-compressed 180B MoE model for Blackwell
2 min readupdated
This is a quantized (weight-compressed) build of the 180-billion-parameter Qwen3.8-Flash-Next language model, using an MoE (Mixture-of-Experts, where only part of the network fires per token: 10 out of 512 experts plus 1 shared expert) architecture. Creator Grawnch packed it down to 2.4592 bpw (bits per weight) in a native format for NVIDIA Blackwell GPUs (SM120 Mode-1). The v5.1 release improves accuracy over the earlier v5 build without adding a single byte of memory overhead, noticeably cutting layer-level quantization error. It is aimed at owners of high-end Blackwell workstations and servers who want to run a massive coding and math model within a tight VRAM budget.
Key features
- 62.10% lower Gate/Up error: SwiGLU neuron permutation and scale calibration (EBSS + KBVQ Cholesky GPTQ) sharply reduced average error across all 48 layers with zero increase in model size.
- Refined Down-Proj projections: Ky-Fan-8 and recomputation of 1,152 singular spines (Rank-8 SVD spines) cut VQ4 quantization error by another 10.06%.
- Bundled speculative decoding: Includes recalibrated
DFlash2 SWZ4drafters for faster generation, along with 12 trainedQSAsparse attention indexers. - Hybrid architecture: Combines 36 Gated-DeltaNet layers with 12 Sparse QSA layers.
What you can do
- Tackle complex algorithmic problems: Run competitive programming tasks and long math reasoning chains on local hardware.
- Speed up inference on Blackwell GPUs: Use the bundled
dflash2_swz4drafters for lower-latency speculative text generation.
Example
To offset cumulative logit variance across 48 layers under sub-2.5-bit compression on coding and math tasks, the creator recommends these sampling settings:temperature: 0.6, top_p: 0.95, top_k: 20, min_p: 0.0, presence_penalty: 0.0.
Read more
DetailsMean Hessian error reduction: -62.10% for Gate/Up, -10.06% for Down-Proj VQ4, and -2.71% validation MSE for the shared expert (shexp_dn).
Container format: SM120 Mode-1 (SWZ2_G64 + VQ4 + Rank-8 whitened SVD spines + SWZ8_G32), 2.4592 bpw.
Requirement: You also need to download the second shard containing global tensors,
Qwen3.8-Flash-Next-WV-v5-3.5bpw-00002-of-00002.gguf, along with the config files from incoai/Qwen3.8-Flash-Next-DFlash2.









Comments