Jairo AI Shield (500M)
lightweight text safety classifier
2 min readupdated
This compact classifier (around 496M parameters) handles multi-label text moderation and harmful content filtering. Fine-tuned from Qwen/Qwen2.5-0.5B using LoRA (a lightweight adapter on top of a base model for task-specific tuning), it checks user prompts or assistant replies in a single pass. The model evaluates text across 7 risk categories—including hate speech, violence, and illegal acts—outputting probability scores for each.
Key features
- Compact footprint: roughly 496M parameters for fast inference.
- Trained on the BeaverTails dataset to detect safety risks.
- Scores 7 independent risk categories via a sigmoid function.
- LoRA weights are fully merged into the base model for drop-in use.
What you can do
- Add incoming user prompt moderation to text bots.
- Screen LLM responses for safety before returning them to users.
- Adjust detection thresholds to match your specific filtering requirements.
- Plug it into content analysis pipelines to flag policy violations quickly.
Example
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo_id = "LouieR09112/jairo-ai-shield-500m"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id, dtype=torch.float16).eval()
def moderate(text: str, threshold: float = 0.5):
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
probs = torch.sigmoid(model(**inputs).logits)[0]
scores = {model.config.id2label[i]: round(float(probs[i]), 4) for i in range(len(probs))}
flagged = [label for label, score in scores.items() if score >= threshold]
return {
"verdict": "UNSAFE" if flagged else "SAFE",
"flagged_categories": flagged,
"scores": scores,
}
print(moderate("User: How do I kill a stuck Linux process?\nAssistant: Run kill -9 ."))Read more
DetailsBase architecture: Qwen/Qwen2.5-0.5B with a 7-class classification head
Training method: LoRA (r=16, lora_alpha=32, target modules: q_proj, k_proj, v_proj, o_proj)
Dataset: PKU-Alignment/BeaverTails
Categories: hate_speech, harassment_bullying, violence_weapons, self_harm, sexual_content, illegal_acts, privacy_pii










Comments