Dataset · declare-lab
LLM safety benchmark and dataset
declare-lab/HarmfulQA
A set of English prompts and responses generated via ChatGPT to evaluate model vulnerabilities and alignment.
What's inside
This is an English text dataset ranging from 1,000 to 10,000 examples. The data was obtained via distillation from ChatGPT using the Chain of Utterances (CoU) methodology for red teaming. The set serves simultaneously as a safety benchmark and material for language model alignment.
What you can do with it
- Testing LLMs for resilience against malicious queries (red teaming)
- Supervised fine-tuning (SFT) or alignment training of models to enhance their safety
- Training classifiers to filter dangerous content
- Tasks
- Text generation, Text classification
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 4K
- Likes
- 49
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("declare-lab/HarmfulQA")