Dataset · wmt
WMT14 classical multilingual machine translation corpus
wmt/wmt14
A large parallel text corpus in six languages, including Russian, for training translation systems.
What's inside
The set contains between 10M and 100M text examples in English, Russian, German, French, Czech, and Hindi, collected for the WMT 2014 competition. The card includes an important warning: some Common Crawl data contains unaligned sentences and English insertions in foreign-language files.
What you can do with it
- Training and comparing machine translation models (including for the EN-RU language pair)
- Developing cleaning and filtering algorithms for noisy parallel corpora
- Tasks
- Translation
- Languages
- Czech, German, English, French, Hindi, Russian
- Size
- 10M–100M rows
- Downloads this month
- 9.8K
- Likes
- 39
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("wmt/wmt14")