KOS-V5.1-Instruct
an LLM built with weight-space programming
2 min readupdated
Built by researchers from the University of Kentucky and the University of Louisville, this 3.72-billion-parameter model was created using weight-space programming. Instead of standard training, its weights were produced through arithmetic operations on task vectors applied to the network layers. This release fixes issues from the base KOS-V5-Instruct version, including a RoPE (rotary position embedding) configuration flaw, which significantly improves long-context handling. The model is aimed at research tasks and advanced tool use.
Key features
- Built via task-vector arithmetic without training runs
- Fixed RoPE configuration (θ = 25,000) to restore long-context support
- Major gains on BFCL benchmarks and reasoning tasks (BBH, ASDiv)
- Accurate self-identity recognition instead of falsely claiming OpenAI authorship
What you can do
- Run the model locally using the transformers library in bfloat16
- Serve it with vLLM using a native context length of up to 24,576 tokens
- Use built-in function calling via native tags
- Apply it to research tasks requiring strict instruction following and tool use
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Kentucky-Open-Science/KOS-V5.1-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "Write a haiku about Kentucky. Do not use any commas."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))Read more
DetailsArchitecture: Decoder-only Transformer (Qwen3ForCausalLM) with GQA
Parameters: 3.715 B, hidden size 2560, 36 layers
Precision: bfloat16 (around 7.43 GB VRAM)
Limitations: research use only; do not use in commercial or medical production systems










Comments