Grace-G1-350M-Instruct: A compact bilingual LLM with a hybrid architecture
1 min readupdated
Grace-G1-350M-Instruct is an instruction-tuned version of a 377M parameter model by BranchingNLP, built for conversations, small talk, and simple tasks in English and Italian. Trained from scratch with a hybrid Gated DeltaNet architecture and the Muon optimizer, it is a great fit for researchers and developers interested in compact architectures, though its small size comes with limitations and a tendency to hallucinate on complex topics.
Key features
- 377M size: A compact language assistant for research tasks.
- Bilingual: Supports Italian and English.
- Hybrid architecture: Uses Gated DeltaNet (with a 3:1 ratio) and gated attention.
- Muon optimizer: An unconventional approach to model training.
What you can do
- Conversations and small talk: Chat with the model on general topics in English or Italian.
- Basic tasks: Run simple text prompts, tell jokes, or solve elementary arithmetic.
- Script integration: Use the model for local experiments thanks to low system requirements.
Example
Running model inference using the script from the repository:
python3 chat_grace.py --model . --temp 0.8 --rep-penalty 1.3 --no-historyRead more
DetailsBenchmarks: A research-grade model; knowledge and reliability are limited by the 377M size.
Hardware requirements: Requires a GPU and the fla library with Triton kernels.
Context and details: 2048 token context window, 32768 vocabulary, 24 layers, 1024 d_model.
License: Apache-2.0.










Comments