Dataset · Helsinki-NLP
Parallel UN document corpus in 6 languages
Helsinki-NLP/multiun
A massive multilingual corpus of official UN document translations for training machine translation systems.
What's inside
It contains parallel texts in six languages: Russian, English, German, Spanish, French, and Arabic. The data volume ranges from 100 million to 1 billion examples. Materials are sourced from the UN website for the period from 2000 to 2009, cleaned, and converted into XML format by specialists from the German center DFKI.
What you can do with it
- Training and fine-tuning machine translation models (including pairs with Russian)
- Aligning multilingual embeddings and cross-lingual search
- Testing translation quality on official and legal vocabulary
- Tasks
- Translation
- Languages
- Arabic, German, English, Spanish, French, Russian, Chinese
- Size
- 100M–1B rows
- Downloads this month
- 5.7K
- Likes
- 15
How to load it
from datasets import load_dataset
ds = load_dataset("Helsinki-NLP/multiun")