Qwen3.8-Flash-Next Gyro-Coder
a 125B coding model on a single 24 GB GPU
3 min readupdated
Qwen3.8-Flash-Next Gyro-Coder is a specialized GGUF build of the massive Qwen/Qwen3.8-Flash-Next language model (125B parameters, MoE — mixture-of-experts architecture) by the agentionai team. The developers pruned half of the experts, keeping only the 256 out of 512 per layer dedicated to programming, and compressed the weights with a custom trellis coder down to 2.125–3.2 bits per weight. As a result, the model fits onto a single consumer GPU like an RTX 3090, 4090, or 7900 XTX while maintaining near-identical token predictions to the original on coding tasks. It is built for developers who want a heavy-hitting local assistant for writing, refactoring, and debugging code.
Key features
- Smart expert pruning: evolutionary selection leaves only the 65B active parameters critical for programming in GPU memory.
- Two variants for different VRAM capacities: the base
TQ2_0takes up 19.92 GiB (fitting into 24 GB even with a 64k context), whileGyro-Coder-L(TQ3_0) requires 27.54 GiB for 32 GB cards. - Reliable tool calls (v1.2): the updated release restored three key experts, completely eliminating broken tags during tool calls in agentic sessions.
- Lightweight downloads (
-noPLE): the n-gram table (PLE) can be downloaded separately once, shrinking the model file size from 75.8 GB down to 21.4 GB.
What you can do
- Hook up to an IDE or agentic framework: spin up a local OpenAI-compatible API and point your editor plugin to it for autocomplete and multi-file edits.
- Automate refactoring and bug fixing: feed the model stack traces, SQL queries, or C++, Python, TypeScript, and Rust code with clear instructions.
- Run inside coding agents with tool calls: Gyro-Coder-L v1.2 completed 292 tool calls across agentic tasks without a single formatting error.
Example
hf download agentionai/Qwen3.8-Flash-Next-Gyro-Coder-GGUF Qwen3.8-Flash-Next-Gyro-Coder-TQ2_0.gguf --local-dir ~/models
docker run --rm -it --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all \
-v ~/models:/models -p 8080:8080 agentionai/agention-llama:server \
-m /models/Qwen3.8-Flash-Next-Gyro-Coder-TQ2_0.gguf -ngl 999 -fa on --jinja --ngram-on-disk -c 65536
Read more
DetailsQuality: top-1 agreement with the full Q8_0 model reaches 94.5% on C++ and 93.1% on Python (95.5% and 94.0% for Gyro-Coder-L); passes 13 out of 16 automated benchmark tasks.
Hardware and requirements: the 19.92 GiB weights (21.1 GiB with a 64k context in
q8_0) require a 24 GB VRAM GPU and a fast NVMe SSD for offloading the n-gram table (--ngram-on-disk).Limitations: stock
llama.cpp, Ollama, and LM Studio do not support this format yet — you need the fork from agentionai or strix-llama.cpp; general knowledge and character tables (such as the Greek alphabet) are degraded due to pruning half the experts.









Comments