Skip to content

Dataset · cais

MMLU: Massive multitask benchmark

cais/mmlu

Multiple-choice QA benchmark across 57 subjects in English.

What's inside

MMLU is a massive multitask test consisting of multiple-choice questions covering humanities, social sciences, STEM, and other fields. Spans 57 subjects; language is English, format is text. Dataset from the paper Measuring Massive Multitask Language Understanding (ICLR 2021).

What you can do with it

  • Evaluating language model knowledge
  • Few-shot LLM benchmarking
  • Comparing models across subjects
Tasks
Question answering
Languages
English
Size
100K–1M rows
Downloads this month
612.7K
Likes
898

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("cais/mmlu")