Dataset · wmt
WMT English-German parallel corpus
wmt/wmt_t2t
Classic corpus of millions of English-German sentence pairs for machine translation tasks.
What's inside
A text dataset containing between 1M and 10M records prepared for the Tensor2Tensor library using data from statmt.org. It contains parallel texts in English and German from various years and sources. It also allows assembling custom datasets for other WMT language pairs.
What you can do with it
- Training and fine-tuning machine translation models (English-German pair)
- Evaluating translation system quality on standard WMT data
- Pre-training bilingual language models
- Tasks
- Translation
- Languages
- German, English
- Size
- 1M–10M rows
- Downloads this month
- 22.8K
- Likes
- 1
How to load it
from datasets import load_dataset
ds = load_dataset("wmt/wmt_t2t")