Qwen3.6-35B-A3B-REAP25
lightweight Qwen3.6 variant with pruned MoE experts
2 min readupdated
A modified language model by inference-optimization, built on Qwen/Qwen3.6-35B-A3B. It removes 25% of the original 256 experts across each of the 40 MoE (Mixture of Experts — an architecture where only specific parts of the network process each token) layers, leaving the 192 most important experts per layer. Pruning was performed using the REAP method (activation-based expert importance scoring) and the llmcompressor tool, with calibration done on the RazorCal dataset. The model targets researchers and developers looking to lower hardware requirements and speed up inference on a large model without retraining weights.
Key features
- Reduced expert count: removes the 64 least important experts in each of the 40 layers (leaving 7,680 out of 10,240 in total).
- Preserved weights: remaining weights (attention, vision module, embeddings, and shared experts) stay in the original BF16 (high-precision floating-point format) precision.
- Unchanged MTP: the multi-token prediction (MTP) layer is left uncalibrated and untouched.
- Automatic re-indexing: the model configuration (`config.json`) is updated with the new expert count, and expert indices are renumbered sequentially.
What you can do
- Run in pipelines: integrate the model into projects via the
transformerslibrary to test text generation workloads. - Benchmark performance: measure generation speed and memory usage against the base Qwen3.6-35B-A3B model.
- Run compressed inference: deploy the model in environments that support modified MoE layer configurations.
Example
from transformers import AutoProcessor, Qwen3_5MoeForConditionalGeneration
model = Qwen3_5MoeForConditionalGeneration.from_pretrained(
"inference-optimization/Qwen3.6-35B-A3B-REAP25", torch_dtype="bfloat16"
)
processor = AutoProcessor.from_pretrained(
"inference-optimization/Qwen3.6-35B-A3B-REAP25"
)
Read more
DetailsBase model: Qwen/Qwen3.6-35B-A3B
Parameter count: ~27.89B
Optimization method: REAP (25% expert pruning)
Calibration dataset: RazorCal (2,048 samples)
Library: transformers










Comments