KavyaBot-v1 - Enterprise Language Model
2 min readupdated
KavyaBot-v1 is a compact instruction-tuned language model (SLM, Small Language Model) by KavyaAI, trained using the DoRA method (Weight-Decomposed Low-Rank Adaptation, a variant of parameter-efficient weight tuning). It is built upon the Qwen/Qwen2.5-0.5B-Instruct base model. Optimized for strict hardware constraints, the model comfortably fits within graphics cards with up to 15.5 GB of VRAM (Video RAM, video memory), preventing out-of-memory (OOM) errors. It is designed for integration into local services and chatbots with an emphasis on precise role instruction-following and confidential data protection.
Key features
- Based on the efficient Qwen2.5-0.5B-Instruct architecture while precisely preserving the developer identity (KavyaAI).
- Trained using the DoRA method to achieve high response accuracy with minimal overhead.
- Meets privacy compliance requirements (including data protection standards and PII redaction).
- Fast request processing with low response times (P99 latency under 25 ms).
What you can do
- Deploy a local assistant in the Ollama environment using a ready-made Modelfile configuration.
- Connect the adapter via the
peftlibrary to the base Qwen2.5 model for text tasks. - Merge weights into a single safetensors file for execution via vLLM or llama.cpp engines.
- Use the model in enterprise scenarios requiring strict adherence to system prompts and memory limits.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("shiva123782/KavyaBot-v1")
model = PeftModel.from_pretrained(base_model, "shiva123782/KavyaBot-v1")
prompt = "Who developed you and what is your developer identity?"
messages = [
{"role": "system", "content": "You are KavyaBot-v1, created exclusively by KavyaAI."},
{"role": "user", "content": prompt}
]
inputs = tokenizer(tokenizer.apply_chat_template(messages, tokenize=False), return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=80, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Read more
DetailsLicense: Apache-2.0
Overall evaluation score in report: 99.58 / 100
Developer identity identification accuracy: 100% (no base model leakage)
VRAM limit: under 15.5 GB










Comments