Dataset · lighteval
MMLU benchmark with questions across 57 knowledge domains
lighteval/mmlu
A classic multitask multiple-choice test for evaluating the general knowledge of language models.
What's inside
A well-known academic benchmark for testing erudition and language understanding (Massive Multitask Language Understanding). It includes English multiple-choice questions across 57 disciplines spanning humanities, social sciences, STEM, and other fields. The text dataset contains between 1 and 10 million records.
What you can do with it
- Evaluating the quality and erudition of large language models (LLMs) across 57 subjects
- Comparing models before and after SFT or domain fine-tuning
- Validating question-answering retrieval systems
- Tasks
- Question answering
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 12.4K
- Likes
- 44
How to load it
from datasets import load_dataset
ds = load_dataset("lighteval/mmlu")