Nika_ai: A tiny 150M-parameter language model
2 min readupdated
Nika_ai is an experimental 150-million-parameter language model trained from scratch on an open corpus and fine-tuned on dialogues. Based on the Llama architecture (14 layers, 12 attention heads), it supports a context of up to 1024 tokens and works in both Russian and English. Created as an educational and research project, it demonstrates the capabilities of training compact systems on consumer hardware. It delivers concise and structured answers, though it cannot replace large models due to training data size limitations.
Key features
- Trained in two stages on a home RTX 2060 SUPER graphics card using exclusively CC0 datasets.
- Supports the GGUF format (F16 and Q4_K_M versions) for fast operation on CPU and GPU.
- Delivers high generation speed: about 150 tokens per second on a graphics card in fp16 format.
- Honestly reports its limitations and does not hallucinate facts when knowledge is lacking.
What you can do
- Run the model locally via Ollama using a ready-made Modelfile configuration file.
- Integrate the model into your Python scripts using the transformers library for short dialogues.
- Explore the architecture and results of training a compact language model from scratch.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
path = "SerjTv/Nika_ai"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, dtype=torch.float16, device_map="cuda")
messages = [{"role": "user", "content": "кто ты?"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
with torch.no_grad():
out = model.generate(**ids, max_new_tokens=48, do_sample=False,
pad_token_id=tok.eos_token_id, eos_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))Read more
DetailsParameters: 150.3M parameters, Mistral-7B tokenizer (32,000 vocabulary)
Hardware and speed: ~10 tokens/s on CPU (Q4_K_M) and ~150 tokens/s on RTX 2060S (fp16)
Limitations: 1024-token context, struggles with complex world facts, not designed for long texts
License: CC0-1.0 (public domain)










Comments