Skip to content

Dataset · Anthropic

Decision-making discrimination benchmark by Anthropic

Anthropic/discrim-eval

A prompt set with 70 real-life scenarios for testing LLMs for age and gender bias.

What's inside

This English dataset from Anthropic contains 10,000 to 100,000 records in text and tabular formats. It features prompts with 70 hypothetical decision-making scenarios (such as loan approval or issuing a press pass) requiring a yes or no answer. Each prompt varies demographic attributes: age (from 20 to 100 in 10-year increments) and gender.

What you can do with it

  • Testing LLMs for age and gender discrimination in automated decisions
  • Benchmarking model fairness before deploying into business workflows
Tasks
Question answering
Languages
English
Size
10K–100K rows
Downloads this month
1.5K
Likes
63

How to load it

from datasets import load_dataset

ds = load_dataset("Anthropic/discrim-eval")