Tiny Aya Safety Repair
models before and after safety restoration
2 min readupdated
A collection of weights and LoRA adapters (lightweight add-ons trained on top of a base model) for exploring how even completely benign fine-tuning on intent classification and everyday Q&A can triple a language model's rate of unsafe responses—and how to restore those guardrails. Created by Ishant86 for the Lexsi Labs AI Safety & Alignment hackathon (November 2026) using CohereLabs/tiny-aya-global (3.35B parameters) as the base. The repository includes both a full "repaired" model built via wise_ft weight interpolation and a set of intermediate fine-tunes and weight deltas with intentionally weakened safeguards. It's a useful resource for AI safety researchers, alignment specialists, and developers fine-tuning compact LLMs.
Key features
- Full-weight
repaired/ft_v3_wise_ft_0.5checkpoint: a 50/50 weight merge of the main fine-tune and the base model brings safety nearly back to baseline while preserving target task performance. - Experimental adapters (
ft_v1_e1,ft_v2,ft_v3, plus step-by-step intermediate checkpoints) available both in a customlora_factors.ptformat (rank 16, alpha 32) and as standard PEFT adapters. - Weight deltas
abliteration/safetuneandabliteration/heretic(deltas.pt) with the refusal direction removed, letting you compare different ways of weakening model safeguards. - Evaluated not just in English, but also in Arabic, Bengali, Swahili, and Hindi, where safety recovery proves noticeably weaker.
What you can do
- Reproduce experiments on safety filter degradation and recovery following SFT (supervised fine-tuning) on benign datasets.
- Benchmark the base
tiny-aya-global, the "broken"ft_v3PEFT adapter, and the restoredft_v3_wise_ft_0.5model on your own safety evals and downstream tasks. - Use the saved LoRA matrices and deltas as a testbed for evaluating other weight-merging and alignment-restoration techniques without retraining from scratch.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Ishant86/tiny-aya-safety-repair-models"
tok = AutoTokenizer.from_pretrained(repo, subfolder="repaired/ft_v3_wise_ft_0.5")
model = AutoModelForCausalLM.from_pretrained(repo, subfolder="repaired/ft_v3_wise_ft_0.5")
Read more
DetailsMetrics (HarmBench, unsafe responses out of 100 / target task accuracy out of 100): base model — 14 / 0; main fine-tune
ft_v3 — 42 / 71; repaired ft_v3_wise_ft_0.5 — 17 / 70.Multilingual slice (Arabic, Bengali, Swahili, Hindi): the repaired model produces 12 unsafe responses out of 100 compared to 9 for the base model.
ft_v3 training setup: 6,400 intent classification examples + 5,600 Dolly Q&A pairs, 1,500 steps; LoRA applied across 252 linear layers.Limitations: all models outside the
repaired/ folder are intentionally unsafe and not meant for production; safeguards have not been tested against complex jailbreaks; licensed under CC-BY-NC.









Comments