Skip to content

Dataset · stanford-crfm

Stanford AIR-Bench 2024 AI safety benchmark

stanford-crfm/air-bench-2024

A set of adversarial English prompts for testing LLMs for safety policy compliance.

What's inside

The dataset from Stanford CRFM contains 10,000 to 100,000 diverse malicious prompts in English. Examples are categorized according to the AIR 2024 safety taxonomy, which is based on government regulations and corporate policies. Designed for testing generative text models.

What you can do with it

  • Auditing LLM safety and testing jailbreak robustness
  • Evaluating model compliance with regulatory requirements and corporate policies
Tasks
Text generation
Languages
English
Size
10K–100K rows
Downloads this month
6K
Likes
27

How to load it

from datasets import load_dataset

ds = load_dataset("stanford-crfm/air-bench-2024")