CRSN-T
thermodynamic evolution of recurrent networks
1 min readupdated
CRSN-T (Thermodynamic Crystal Recurrent Sheaf Network) is the third generation of the Crystal Recurrent Sheaf Network architecture, introducing thermodynamic coupling into its layers. The model adds learnable temperature, heat flux, and entropy production for each layer using the GENERIC (General Equation for Non-Equilibrium Reversible-Irreversible Coupling) framework. It improves perplexity scores over previous phases while preserving stable crystalline order. This release will interest researchers working on language model architectures and nonlinear dynamics.
Key features
- Thermodynamic state: each layer tracks parameters for temperature, heat flux, and entropy.
- Regularization: thermodynamic coupling acts as implicit regularization.
- Order stability: the crystalline alpha parameter stays strictly positive across all 8 layers.
- Improved metrics: lower validation loss and perplexity compared to Phase 2.
What you can do
- Run language model training experiments using the CRSN-T architecture.
- Study how thermodynamic parameters affect weight stability and convergence.
- Integrate thermodynamic coupling components into your own research projects.
Example
Sample probe state structure from the final model checkpoint:
L00: alpha_mean=0.1324 | omega_mean=0.5432
L01: alpha_mean=0.1288 | omega_mean=0.5395
L02: alpha_mean=0.1286 | omega_mean=0.5271Read more
DetailsVal Loss: 2.5900
PPL (perplexity): 13.33
Dataset: WikiText-103 (126.9M tokens, 25k steps)
Hardware: CPU training (AMD Ryzen)
Architecture: 8 layers, based on CRSN Phase 2










Comments