Qwen3.5-0.8B-NKC
modified weights for logic and physics
2 min readupdated
Qwen3.5-0.8B-NKC builds on the compact Qwen/Qwen3.5-0.8B language model (873.4 million parameters) using the Neural Knowledge Compiler (NKC) method, which compiles skills from other models directly into existing weights without adding new parameters. By modifying just 0.38% of the weights, the developers significantly boosted its mathematical thinking mode and gave it a basic grasp of scene depth and motion physics. The model loads with the standard transformers library and requires no custom inference code, making it useful for developers and researchers who need a lightweight model with stronger reasoning capabilities.
Key features
- Zero parameter overhead: All upgrades are written directly into the base model's existing tensors.
- Major math gains: In thinking mode, scores jumped from 6% to 53% on MATH-500 and from 16% to 76% on GSM8K.
- Added depth estimation: Integrating data from Depth Anything V2 improved its ability to judge object distance.
- Physics prediction: Trained on JEPA-WM data to predict whether a push will move an object.
What you can do
- Solve math and logic problems with high accuracy using thinking mode (
enable_thinking=True). - Test spatial perception and depth estimation on images using the built-in vision layers.
- Drop the model into local chatbots and text-and-image pipelines via a standard
transformerssetup.
Example
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("MichielH/Qwen3.5-0.8B-NKC", dtype="bfloat16", device_map="auto")
processor = AutoProcessor.from_pretrained("MichielH/Qwen3.5-0.8B-NKC")
messages = [{"role": "user", "content": [{"type": "text", "text": "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}."}]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Read more
Details• Base model: Qwen/Qwen3.5-0.8B (873.4M parameters)
• Modified weights: ~3.28M (0.38% of total)
• Benchmarks: MATH-500 (with thinking) jumped from 6% to 53%; GSM8K rose from 16% to 76%
• Limitations: Math performance may dip slightly in default mode without thinking enabled










Comments