DuoNeural/RMSN-229M-ArnoldTongue-Omega45-Base
a resonant language model
2 min readupdated
DuoNeural’s experimental 229M-parameter neurodynamic language model is the first practical exploration of Arnold tongue resonance in training neural architectures. Built on the Resonant Metriplectic Sheaf Network (RMSN) architecture, it hit a record validation perplexity (ppl) of 49.8 on the WikiText-103 dataset for the author. Under the hood, it runs a hidden dimension of 1024, 20 layers, and 64 Poincaré working slots. It is aimed at architecture researchers and anyone interested in unusual topological approaches to LLM training.
Key features
- Resonant geometry: uses a resonance ratio of Ω = 4/5 (Arnold tongue q=5) to maintain topological locking during training.
- Record perplexity: the best checkpoint (at 80k steps) reached 49.8 ppl — DuoNeural’s first result under 50.
- Topological phases: training exhibited two cascading reorganization phases of the soliton lattice.
- Consumer hardware training: trained on a single NVIDIA RTX 5070 Ti (15.5 GB VRAM).
What you can do
- Run local inference using the custom PyTorch configuration code.
- Reproduce the Arnold tongue resonance experiment with the training scripts in the repository.
- Probe the percolation sheaf topology of the trained model using probe_percolation_sheaf.py.
Example
Loading the model weights and RMSN configuration in PyTorch:
import torch
from rmsn_core import RMSNConfig, RMSNForCausalLM
ckpt_path = "rmsn_best.pt"
checkpoint = torch.load(ckpt_path, map_location="cuda:0")
config = RMSNConfig(
vocab_size=49152,
hidden_dim=1024,
num_layers=20,
num_workspace_slots=64,
max_seq_len=512
)
model = RMSNForCausalLM(config)
model.load_state_dict(checkpoint["model_state"])
model.eval().cuda()Read more
DetailsParameters: 229.17M | Hidden dimension (D): 1024
Depth (L): 20 layers | Slots (M_w): 64 Poincaré slots
Resonance ratio: 4/5 (q=5) | Training tokens: 126.9M (WikiText-103)
Best Val PPL: 49.8 (80k steps) | Final Val PPL: 51.5 (100k steps)
Hardware: NVIDIA RTX 5070 Ti (15.5 GB VRAM)










Comments