Artha-1.1B v2
Compact prompt-to-math notation translator
2 min readupdated
The model is a LoRA adapter (a small add-on layer for a base model) for TinyLlama-1.1B-Chat-v1.0 that translates standard English text prompts into a compact Artha notation. Version v2 expands the tokenizer (the model's vocabulary) with 67 special symbols and atoms, reducing prompt token usage by 35.4% compared to English. This development helps optimize workflows for compact language models and saves context space.
Key features
- New tokenizer: 67 frequently used Artha symbols added as whole tokens.
- Token savings: 35.4% reduction in token consumption compared to English text.
- Accuracy: 100% exact match with reference results on the test pattern set.
- Base architecture: built on TinyLlama-1.1B using LoRA (r=16, alpha=32).
What you can do
- Prompt compression: use the model to convert long instructions into the compact Artha format before sending them to the pipeline.
- Pipeline integration: embed the model into local workflows to reduce context length in text processing tasks.
- Notation experiments: explore the potential of mathematical notation for controlling text generation.
Example
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
REPO = "siddsukh/artha-1.1b-v2"
tok = AutoTokenizer.from_pretrained(REPO)
base = AutoModelForCausalLM.from_pretrained(
"TinyLlama/TinyLlama-1.1B-Chat-v1.0", dtype=torch.bfloat16, device_map="auto"
)
base.resize_token_embeddings(len(tok), mean_resizing=False)
model = PeftModel.from_pretrained(base, REPO).eval()
print("Модель успешно загружена")
Read more
DetailsAccuracy (Exact match): 100% on 1,000 test pairs
Token savings: 35.4% compared to standard English text
Training parameters: LoRA r=16, alpha=32, 2 epochs, learning rate 2e-4
Limitations: training was conducted on 9 template task types, tokenizer is specific to this model










Comments