Dataset · CohereLabs
Multilingual dialogue prompt benchmark by Cohere
CohereLabs/aya_evaluation_suite
26,750 open prompts in over 100 languages for evaluating text generation quality.
What's inside
The dataset contains 26,750 open dialogue prompts to test language model performance. Some examples are human-annotated in 7 languages (English, Arabic, Chinese, Portuguese, Turkish, Yoruba, and Telugu), while the rest are obtained via machine translation of selected prompts into 101 languages.
What you can do with it
- Evaluate open text generation quality of multilingual LLMs
- Compare models on low-resource languages
- Tasks
- Text generation
- Languages
- Afrikaans, Albanian, Amharic, Arabic, Azerbaijani, Belarusian, Bangla, Bulgarian and 4 more
- Size
- 10K–100K rows
- Downloads this month
- 2.8K
- Likes
- 55
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("CohereLabs/aya_evaluation_suite")