Skip to content

Dataset · CohereLabs

Multilingual benchmark of examination tests

CohereLabs/include-base-44

22,637 four-choice questions across 44 languages to evaluate LLM knowledge and logic.

What's inside

The dataset contains 22,637 text multiple-choice questions (MCQs) in 44 languages, including Albanian, Arabic, Armenian, Azerbaijani, Belarusian, and Bengali. The tasks are extracted from real academic and professional exams and cover 57 different topics.

What you can do with it

  • Evaluate knowledge and logical capabilities of multilingual LLMs
  • Test models on low-resource languages
  • Compare response quality of language models in a real linguistic environment
Tasks
Multiple choice
Languages
Albanian, Arabic, Armenian, Azerbaijani, Belarusian, Bangla, Basque, Bulgarian and 4 more
Size
10K–100K rows
Downloads this month
14.4K
Likes
53

How to load it

from datasets import load_dataset

ds = load_dataset("CohereLabs/include-base-44")