Laguna-XS.2-FP8
a compact MoE model for agentic coding
2 min readupdated
Laguna XS.2-FP8 is an FP8-quantized (8-bit floating point to save VRAM) version of Poolside's language model, built for software development and autonomous coding agents on local hardware. Under the hood, it uses a Mixture-of-Experts (MoE, where only part of the network activates per token) architecture: out of 33B total parameters, only 3B are active at a time, which speeds up generation. Across 30 of its 40 layers, it uses Sliding Window Attention (512-token sliding window) with per-head gating, while the KV cache is also compressed to FP8 to cut memory overhead. The model supports a context length of up to 131,072 tokens, reasons between tool calls, and fits developers looking for a capable local coding assistant under an open Apache 2.0 license.
Key features
- Efficient MoE architecture: 33B total parameters with 3B active per token (256 experts plus 1 shared) and a 3:1 ratio of SWA to global attention layers.
- Built-in reasoning mode: reasons before calling tools and between steps, with thinking toggleable on a per-request basis.
- Ready for local deployment: despite its 33B size, it fits on a Mac with 36 GB RAM and runs out of the box in vLLM, Transformers, TRT-LLM, Ollama, and mlx-lm.
- Permissive Apache 2.0 license: free for commercial and non-commercial use and modification.
What you can do
- Run a local coding and terminal agent: hook it up in vLLM or Ollama with tool calling to execute bash commands, debug issues, and handle long multi-step tasks across a repo.
- Tune reasoning to the task: keep preserved thinking enabled for complex architecture and refactoring, or toggle
enable_thinkingoff for fast boilerplate generation.
Example
VLLM_USE_DEEP_GEMM=0 vllm serve \
--model poolside/Laguna-XS.2-FP8 \
--tool-call-parser poolside_v1 \
--reasoning-parser poolside_v1 \
--enable-auto-tool-choice \
--default-chat-template-kwargs '{"enable_thinking": true}' \
--kv-cache-dtype-skip-layers $(seq 0 39) \
--moe_backend marlin
Read more
DetailsBenchmarks (for the BF16 version): SWE-bench Verified — 68.2%, SWE-bench Multilingual — 62.4%, SWE-bench Pro — 44.5%, Terminal-Bench 2.0 — 30.1%.
Hardware and caveats: on non-Hopper GPUs, the FP8 KV cache can produce garbled output — disable it in vLLM using the
--kv-cache-dtype-skip-layers flag across all 40 layers.Training: pre-training, post-training, and RL with the Muon optimizer; BF16, NVFP4, and INT4 variants are also available.










Comments