Skip to content

Dataset · CohereLabs

Multilingual visual benchmark from Cohere

CohereLabs/AyaVisionBench

Test set of image-question pairs in 23 languages for evaluating multilingual vision-language models.

What's inside

The benchmark from CohereLabs contains 1k to 10k examples (135 image-question pairs for each of 23 languages across 9 task categories). Metadata lists English, French, German, Spanish, Italian, and Portuguese. To answer each question, the model must analyze the image itself.

What you can do with it

  • Test multilingual vision-language models (VLMs) in real-world scenarios
  • Compare model visual comprehension quality across different languages
Languages
English, French, German, Spanish, Italian, Portuguese, Japanese, Korean and 4 more
Size
1K–10K rows
Downloads this month
806
Likes
38

How to load it

from datasets import load_dataset

ds = load_dataset("CohereLabs/AyaVisionBench")