Caracal Instruct v2
an instruction-tuned LoRA adapter for an African LLM
2 min readupdated
Caracal Instruct v2 is a PEFT LoRA adapter (a lightweight add-on for fine-tuning without modifying the entire model's weights) by theophilusowiti built on top of the Caracal_GPT base language model. It turns the base model into a conversational assistant capable of following instructions in the ChatML format. It generates responses across 9 languages out of the box: English, Swahili, Amharic, Luganda, Kinyarwanda, Hausa, Yoruba, Igbo, and Lingala. Ideal for developers building multilingual chatbots and researchers working with low-resource African languages.
Key features
- Supports 9 languages: English, Swahili, Amharic, Luganda, Kinyarwanda, Hausa, Yoruba, Igbo, and Lingala.
- Optimized for dialogue and instruction following using the ChatML template.
- Trained with SFT (Supervised Fine-Tuning) via the
trllibrary on theCaracal_Instruct_Datasetdataset. - Released under the open Apache 2.0 license, allowing free and commercial use.
What you can do
- Integrate it into a chatbot or helpdesk service to communicate with users in African languages and English.
- Use it in NLP research as a ready-to-go baseline adapter for evaluating generation quality on low-resource languages.
- Further fine-tune it for specialized downstream tasks on top of the
Caracal_GPTandCaracal_instruct_v2stack via thepeftlibrary.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("theophilusowiti/Caracal_GPT")
base = AutoModelForCausalLM.from_pretrained("theophilusowiti/Caracal_GPT", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "theophilusowiti/Caracal_instruct_v2")
inputs = tok.apply_chat_template([{"role": "user", "content": "Who is the president of Kenya?"}], return_tensors="pt", add_generation_prompt=True).to(model.device)
print(tok.decode(model.generate(inputs, max_new_tokens=100)[0], skip_special_tokens=True))
Read more
DetailsArchitecture: Causal Language Model with a PEFT LoRA adapter based on
theophilusowiti/Caracal_GPT.Training:
SFTTrainer from trl on the theophilusowiti/Caracal_Instruct_Dataset dataset.Metrics and VRAM requirements are not specified in the model card (training logs were tracked in Weights & Biases).










Comments