Skip to content

Dataset · CohereLabs

Multilingual MMLU in 42 Languages

CohereLabs/Global-MMLU

MMLU questions in 42 languages with machine translation, professional editing, and cultural sensitivity markup.

What's inside

The dataset combines MMLU questions in 42 languages, including English. Some translations are machine-generated, while others are professionally done and supplemented with crowdsourced post-editing. For a subset of 2,850 questions per language, cultural sensitivity annotations are available: Culturally Sensitive or Culturally Agnostic. The format is text.

What you can do with it

  • Evaluating multilingual LLMs on MMLU
  • Researching benchmark cultural sensitivity
  • Testing models in 42 languages
Languages
English, Arabic, Bangla, Spanish, French, Hindi, Russian, German and 4 more
Size
100K–1M rows
Downloads this month
55.6K
Likes
169

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("CohereLabs/Global-MMLU")