Qwen3-8B-InWeight-FPGA-WSP
compiling a dialectical automaton into model weights
2 min readupdated
This model is a special version of Qwen/Qwen3-8B with built-in dialectical logical inference mechanisms (Pure WSP). The creators compiled a directed acyclic graph of a finite automaton directly into the base model's neurons in just 30 seconds without traditional gradient training. This approach alters model behavior at the weight level, improving context retention and logical stability in dialogues. The development is intended for researchers and developers who need enhanced LLM (large language model) reasoning stability without cumbersome external wrappers.
Key features
- Direct compilation into weights (Pure WSP) modifying only 0.035% of parameters without gradient-based training.
- Precise modification of key attention layers and SwiGLU channels to suppress KV cache drift.
- Significant boost in context retention metrics and stability in multi-step dialogues based on benchmarks.
What you can do
- Integrate the model via standard
transformersorvLLMlibraries to conduct dialogue stress tests. - Apply a compact bitstream delta to quickly patch the original
Qwen/Qwen3-8Bweights in milliseconds. - Use in research projects exploring internal weight addressing mechanisms and logical automata in transformers.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "masstensor/Qwen3-8B-InWeight-FPGA-WSP"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
Read more
DetailsBenchmark Suite 1 (Synthesis Acc): 79.36% (+7.95% over base)
Context retention (Holding Rate): 93.64% (+36.99%)
Requirements: graphics card supporting bfloat16, base Qwen3-8B model
Features: modified ~0.035% of parameters in layers 24, 26, 28, 30, and 35










Comments