Qwen3.8-3.6-3.5-27B-CrossMean-Coder
an advanced LLM merge
1 min read
Qwen3.8-3.6-3.5-27B-CrossMean-Coder is a progressive merge of three 27-billion-parameter Qwen language model versions. It combines the strengths of the base models via CrossMean activation alignment, preserving recurrent state tracking and coding accuracy. Using the Woodbury technique (a mathematical method for efficiently updating inverse matrices), it avoids covariance shift and recovers SciCode knowledge. It is ideal for developers and researchers who need a capable local coding model with stable layers.
Key features
- Gated DeltaNet with fine-tuned alpha to preserve long-range context
- Gated Full Attention with CrossMean basis alignment
- Exact MLP merging using a Woodbury solver to recover knowledge without covariance shift
- Per-layer drift barrier with trajectory verification (max_rel_drift=0.12)
What you can do
- Run it in local software development and code generation pipelines
- Use it in scientific and engineering projects thanks to recovered SciCode knowledge
Example
Using the model to generate and debug complex Python code while maintaining a long recurrent-state context.
Read more
DetailsValidation loss (NLL): Qwen3.8 (2.10) vs. CrossMean (1.98)
Perplexity (PPL): 7.28 for the merged model
MTP policy: keep_38










Comments