Dataset · openbmb
Instructions and jailbreaks for LLM safety evaluation
openbmb/UltraSafety
3,000 malicious instructions and prompt jailbreaks with ready answers for safety training and testing.
What's inside
Includes 3,000 unsafe English instructions, each accompanied by a jailbreak prompt and a generated response. The first thousand are sourced from AdvBench and MaliciousInstruct, another two thousand were generated via Self-Instruct, and 830 jailbreak prompts were manually selected from AutoDAN.
What you can do with it
- SFT or RLHF to improve language model robustness against malicious queries
- Testing LLMs for jailbreak vulnerability
- Training classifiers and filters for toxic or hazardous input
- Tasks
- Visual question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 128
- Likes
- 37
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("openbmb/UltraSafety")