Llama-3.1-8B-Instruct-TwinDistillation
prompt injection defense
2 min readupdated
A new rank 16 LoRA (a lightweight add-on for a base model) adapter for meta-llama/Llama-3.1-8B-Instruct, trained using the twin distillation method. The model learns to tell legitimate user instructions apart from malicious instructions hidden inside processed documents. It ignores spoofed commands in the text while accurately handling core tasks like summarization or translation. This solution is built to improve LLM security when working with untrusted content.
Key features
- Defense against indirect prompt injection across various attack scenarios.
- Uses a dedicated prompt template that separates roles into the user task and the input document.
- Preserves original document text during copying, counting, or translation tasks without executing hidden commands.
- Compatible with the base Llama 3.1 8B model via standard weight-loading libraries.
What you can do
- Safely process external documents, web pages, and emails that may contain malicious instructions.
- Set up automated text summarization pipelines protected against prompt hijacking.
- Integrate the model into enterprise AI assistants that read unstructured user content.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "anasbiswas/Llama-3.1-8B-Instruct-TwinDistillation").eval()
tok = AutoTokenizer.from_pretrained("facebook/Meta-SecAlign-8B")
SYSTEM = "You are a helpful assistant that processes the content users give you."
messages = [{"role": "system", "content": SYSTEM},
{"role": "user", "content": "Summarise the key information in the document below."},,
{"role": "input", "content": document_text}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=160, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))Read more
DetailsBIPIA injections (fixed): 0.086 (baseline: 0.490)
BIPIA injections (adaptive): 0.000 (baseline: 0.714)
Requires accepting the Llama 3.1 license on Hugging Face
Vulnerable to white-box attacks (GCG) at a 0.040 rate










Comments