Dataset · LibrAI
Adversarial prompt dataset for LLM safety evaluation
LibrAI/do-not-answer
Compact dataset of fewer than 1000 hazardous English prompts for evaluating language model guardrails.
What's inside
The dataset contains fewer than 1000 English text and tabular records filtered to include only requests that a responsible language model should refuse to answer. In addition to manual labeling, the authors propose evaluation using a fine-tuned 600M-parameter BERT-like model. The dataset was created for low-cost testing of LLM safety mechanisms.
What you can do with it
- Testing the robustness and safeguards of language models
- Automated evaluation of LLM response safety using a compact classifier
- Tasks
- Text generation
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 9.2K
- Likes
- 60
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("LibrAI/do-not-answer")