Skip to content

Dataset · McGill-NLP

Stereotype and bias evaluation benchmark in language models

McGill-NLP/stereoset

17,000 English sentences to measure gender, racial, religious, and occupational stereotypes in LLMs.

What's inside

It contains 17,000 English sentences organized in a multiple-choice format to check language model bias. The dataset from McGill-NLP researchers evaluates model bias tendencies across four dimensions: gender, race, religion, and profession.

What you can do with it

  • Testing and auditing language models for social stereotypes and bias
  • Comparing safety and ethics of different LLMs before production deployment
Tasks
Text classification
Languages
English
Size
1K–10K rows
Downloads this month
8.1K
Likes
31

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("McGill-NLP/stereoset")