Skip to content

Dataset · declare-lab

LLM safety benchmark and dataset

declare-lab/HarmfulQA

A set of English prompts and responses generated via ChatGPT to evaluate model vulnerabilities and alignment.

What's inside

This is an English text dataset ranging from 1,000 to 10,000 examples. The data was obtained via distillation from ChatGPT using the Chain of Utterances (CoU) methodology for red teaming. The set serves simultaneously as a safety benchmark and material for language model alignment.

What you can do with it

  • Testing LLMs for resilience against malicious queries (red teaming)
  • Supervised fine-tuning (SFT) or alignment training of models to enhance their safety
  • Training classifiers to filter dangerous content
Tasks
Text generation, Text classification
Languages
English
Size
1K–10K rows
Downloads this month
4K
Likes
49

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("declare-lab/HarmfulQA")