Skip to content

Dataset · LibrAI

Adversarial prompt dataset for LLM safety evaluation

LibrAI/do-not-answer

Compact dataset of fewer than 1000 hazardous English prompts for evaluating language model guardrails.

What's inside

The dataset contains fewer than 1000 English text and tabular records filtered to include only requests that a responsible language model should refuse to answer. In addition to manual labeling, the authors propose evaluation using a fine-tuned 600M-parameter BERT-like model. The dataset was created for low-cost testing of LLM safety mechanisms.

What you can do with it

  • Testing the robustness and safeguards of language models
  • Automated evaluation of LLM response safety using a compact classifier
Tasks
Text generation
Languages
English
Size
up to 1K rows
Downloads this month
9.2K
Likes
60

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("LibrAI/do-not-answer")