Dataset · cais
MMLU: Massive multitask benchmark
cais/mmlu
Multiple-choice QA benchmark across 57 subjects in English.
What's inside
MMLU is a massive multitask test consisting of multiple-choice questions covering humanities, social sciences, STEM, and other fields. Spans 57 subjects; language is English, format is text. Dataset from the paper Measuring Massive Multitask Language Understanding (ICLR 2021).
What you can do with it
- Evaluating language model knowledge
- Few-shot LLM benchmarking
- Comparing models across subjects
- Tasks
- Question answering
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 612.7K
- Likes
- 898
Models trained on it
- jondurbin/bagel-dpo-34b-v0.2Text generation 9.2K
- mradermacher/UNA-34Beagles-32K-bf16-v1-i1-GGUF 1.4K
- mradermacher/UNA-34Beagles-32K-bf16-v1-GGUF 1.2K
- itssaharsh/tiny-aya-global-unlearn-bio-circuitkit-layersText generation 1K
- bartowski/bagel-dpo-8x7b-v0.2-GGUFText generation 998
- TheBloke/nontoxic-bagel-34b-v0.2-GGUF 896
- OpenMed/Qwen3-1.7B-WSP-ClinicalText generation 842
- OpenMed/Qwen3-0.6B-WSP-ClinicalText generation 804
How to load it
from datasets import load_dataset
ds = load_dataset("cais/mmlu")