Skip to content

Dataset · gsarti

101-language machine translation benchmark

gsarti/flores_101

Professionally translated Wikipedia sentences in 101 languages for evaluating multilingual translation quality.

What's inside

Inside are 3001 sentences from English Wikipedia on various topics, aligned across all languages. Texts are translated into 101 languages (including rare and low-resource ones) by professional translators under strict quality control. Metadata indicates a total volume from 100 thousand to 1 million records in text and tabular formats.

What you can do with it

  • Evaluating machine translation system quality, especially on low-resource languages
  • Testing multilingual models in many-to-many translation mode
Tasks
Text generation, Translation
Languages
Afrikaans, Amharic, Arabic, Armenian, Assamese, Asturian, Azerbaijani, Belarusian and 4 more
Size
100K–1M rows
Downloads this month
2K
Likes
35

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("gsarti/flores_101")