Skip to content

Dataset · openbmb

Instructions and jailbreaks for LLM safety evaluation

openbmb/UltraSafety

3,000 malicious instructions and prompt jailbreaks with ready answers for safety training and testing.

What's inside

Includes 3,000 unsafe English instructions, each accompanied by a jailbreak prompt and a generated response. The first thousand are sourced from AdvBench and MaliciousInstruct, another two thousand were generated via Self-Instruct, and 830 jailbreak prompts were manually selected from AutoDAN.

What you can do with it

  • SFT or RLHF to improve language model robustness against malicious queries
  • Testing LLMs for jailbreak vulnerability
  • Training classifiers and filters for toxic or hazardous input
Tasks
Visual question answering
Languages
English
Size
1K–10K rows
Downloads this month
128
Likes
37

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openbmb/UltraSafety")