Qwen3.8-27b-Terse-Coder-Basalt
a compact version of Qwen3.8-27B for fast coding
2 min readupdated
A ready-to-run package of Qwen3.8-27B by Shockem, fine-tuned for coding and packaged in the Basalt format (a file format for running quantized models locally). The main highlight of the Terse-Coder version is cutting the chain-of-thought roughly tenfold while maintaining answer accuracy. The model is based on Qwen/Qwen3.8-27B with the Terse-Coder adapter merged in and quantized in mixed precision (NVFP4 and FP8), bringing the file size down to 21.6 GB. It suits developers who need a local 27B-parameter model that doesn't waste extra tokens and time on lengthy reasoning before outputting code.
Key features
- Concise reasoning: uses roughly 10x fewer tokens for chain-of-thought on coding tasks without sacrificing solution accuracy.
- Mixed quantization (21.6 GB): MLP layers and the output layer are compressed to NVFP4, attention projections and GDN to FP8 E4M3, while embeddings remain in BF16.
- Vision and long-context support: the package includes a vision projector, and the model was tested at a 131,072-token context length with an int8 KV cache.
- Tool use: passed an internal benchmark suite (15/15 in 64 seconds) covering instruction following, JSON output, tool calling, and code generation.
What you can do
- Local coding assistant: hook it up to an IDE or agentic pipeline (for agentic mode, the author recommends the
Shockem/froggeric-terse-codertemplate) for fast code generation with minimal token overhead. - Run on dual mid-range GPUs: deploy it across two 16 GB VRAM cards (the model card includes benchmarks on dual RTX 5060 Ti 16 GB GPUs with layer splitting).
- Automation and function calling: use it in local scripts that require strict JSON output, tool calling, and up to 128k context.
Example
basalt serve Qwen3.8-27B-Terse-Coder-NVFP4.basalt --spec 0 --kv int8
Read more
DetailsOriginal checkpoint metrics on vLLM (not re-benchmarked in the Basalt format): HE+ 91.5, MBPP+ 79.4, GSM8K 98.5, GPQA 79.8.
Speed benchmarks on Basalt 0.4.1 (2× RTX 5060 Ti 16 GB, 131,072 context, int8 KV): prefill — 3,491–5,373 tok/s (across prompt lengths from 512 to 64k), generation — 22.1 tok/s.
Limitation: when splitting layers across two GPUs, you must pass the
--spec 0 flag (with spec >= 2, the built-in FP8 MTP drafter crashes with a memory error on the first token).Recommended sampling parameters (baked into the package): temperature 0.6, top_p 0.95, top_k 20. License: Apache 2.0.










Comments