Mesosfer d24 SFT
custom GPT model for instructions, code, and agent tasks
2 min read
A compact language model based on a custom PyTorch GPT architecture (d24), trained via SFT (Supervised Fine-Tuning on prepared example dialogues and tasks). The authors trained it on a combined d24-tulu-nemotron-mix dataset, merging Tülu 3 instruction-following and code-writing datasets with NVIDIA Nemotron-SFT-Agentic-v4 agent data. As a result, the model can chat, write code, and execute multi-step instructions. This solution is well-suited for developers and hobbyists looking for a lightweight local model with a ready-to-run inference pipeline independent of heavy standard frameworks.
Key features
- Trained on a mixture of Tülu 3 Instruction Following, Tülu 3 Code, and NVIDIA Nemotron-SFT-Agentic-v4 datasets for code, instructions, and agent scenarios.
- Built on a custom 24-layer GPT architecture (
n_layer=24,n_head=12,n_embd=1536) with a 2048-token context length. - Uses a custom RustBPE / tiktoken-based tokenizer with a
vocab_size=65536vocabulary. - Delivered in BF16 precision with a file weight of about 5.4 GB, along with a ready-to-use
inference.pyscript for a quick start.
What you can do
- Run a local console chat or single response generation using the built-in
inference.pyscript. - Use it for generating and reviewing basic Python code, such as writing helper functions.
- Test it in prototypes of simple text assistants and agent chains by adjusting
temperatureandrepetition-penaltysettings.
Example
python ~/.cache/mesosfer/inference.py --prompt "Write a Python function to check if a string is a palindrome."
python ~/.cache/mesosfer/inference.py --temperature 0.2 --repetition-penalty 1.08Read more
DetailsCheckpoint: d24-tulu-v1 (step 6431, val_bpb = 0.3940)
Architecture: 24 layers, 12 attention heads (n_kv_head=12), embedding size 1536, context 2048, window_pattern="L"
File weight: ~5.4 GB in BF16 format, running via Mesosfer with CUDA or ROCm is supported
Language: English










Comments