Skip to content

Dataset · CohereLabs

Multilingual dialogue prompt benchmark by Cohere

CohereLabs/aya_evaluation_suite

26,750 open prompts in over 100 languages for evaluating text generation quality.

What's inside

The dataset contains 26,750 open dialogue prompts to test language model performance. Some examples are human-annotated in 7 languages (English, Arabic, Chinese, Portuguese, Turkish, Yoruba, and Telugu), while the rest are obtained via machine translation of selected prompts into 101 languages.

What you can do with it

  • Evaluate open text generation quality of multilingual LLMs
  • Compare models on low-resource languages
Tasks
Text generation
Languages
Afrikaans, Albanian, Amharic, Arabic, Azerbaijani, Belarusian, Bangla, Bulgarian and 4 more
Size
10K–100K rows
Downloads this month
2.8K
Likes
55

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("CohereLabs/aya_evaluation_suite")