ordos-research/nekyia-virgilio
Language model for medieval languages
2 min readupdated
Nekyia: Virgilio is a compact language model with 893.5M parameters, trained from scratch on historical texts in Latin, Italian, Old French, Middle English, and Old Spanish created before 1450. The model is built on the Qwen3ForCausalLM architecture and includes a base version for text generation, as well as a chat fine-tune trained to answer questions in a conversational format. The project focuses on linguistic tasks, historical text research, and working with ancient written sources.
Key features
- Trained on a corpus of about 5.6B tokens without using modern texts or Wikipedia.
- Support for Latin, Italian, Old French, Middle English, and Old Spanish languages.
- Qwen3ForCausalLM architecture with 28 layers and a 2048-token context window.
- Short conversational fine-tune (chat) for generating responses in the style of medieval dialogues and disputes.
What you can do
- Apply in historical projects or chatbots to generate responses in medieval languages.
- Explore linguistic features of texts up to 1450 using a 65,536-token tokenizer.
- Use the base model (via the subfolder="base" parameter) for historical text autocompletion.
Example
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "ordos-research/nekyia-virgilio"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo).eval()
prompt = '<|doc|> lang=la date=1250..1280 id=chat source=scta genre=dialogue\nA. Quid est iustitia?\nM.'
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
newline = tok.encode("\n", add_special_tokens=False)[-1]
out = model.generate(ids.repeat(4, 1), max_new_tokens=100, do_sample=True, temperature=0.8, top_p=0.95,
eos_token_id=[newline, tok.eos_token_id], pad_token_id=tok.pad_token_id)
for o in out:
print(tok.decode(o[ids.shape[1]:], skip_special_tokens=True).split("\n")[0].strip())Read more
DetailsParameters: 893.5 M, layers: 28, width: 1536 / 4096.
Training: 8 NVIDIA H100 GPUs for 14.2 hours (27,100 steps).
Limitations: the model is small, response content may be circular or incorrect, reflects medieval views of sources.
License: cc-by-nc-sa-4.0.










Comments