Skip to content

Dataset · tasksource

Lightweight duplicate-free MMLU benchmark

tasksource/mmlu

Cleaned and fast version of the well-known MMLU benchmark in English for evaluating language model knowledge.

What's inside

This is a repackaged version of the classic English MMLU benchmark (hendrycks_test), from which the auxiliary training set (auxiliary train) was removed. Due to the absence of duplicate data, the size was reduced from 162 MB to 7 MB with a volume of 10,000 to 100,000 examples. The data is intended for multiple-choice, classification, and question-answering tasks.

What you can do with it

  • Rapid evaluation of erudition and general knowledge of language models across dozens of disciplines
  • Integration into LLM automated testing pipelines without excessive disk and memory overhead
Tasks
Text classification, Multiple choice, Question answering
Languages
English
Size
10K–100K rows
Downloads this month
30.8K
Likes
37

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("tasksource/mmlu")