Skip to content

Dataset · openai

MMMLU: MMLU in 14 languages

openai/MMMLU

MMLU test questions across 57 topics translated into 14 languages by professional translators.

What's inside

MMLU is a general knowledge benchmark across 57 categories: from basic level to law, physics, history, and computer science. The test set was translated by professional translators into 14 languages; metadata specifies ar, bn, de, es, fr, hi, and others. Format is question-answer pairs.

What you can do with it

  • Evaluate multilingual LLMs on general knowledge
  • Compare benchmark translation quality
  • Test QA systems on exam-level questions
Tasks
Question answering
Languages
Arabic, Bangla, German, Spanish, French, Hindi, Indonesian, Italian and 4 more
Size
100K–1M rows
Downloads this month
15.4K
Likes
539

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openai/MMMLU")