Skip to content

Dataset · Helsinki-NLP

Parallel UN document corpus in 6 languages

Helsinki-NLP/multiun

A massive multilingual corpus of official UN document translations for training machine translation systems.

What's inside

It contains parallel texts in six languages: Russian, English, German, Spanish, French, and Arabic. The data volume ranges from 100 million to 1 billion examples. Materials are sourced from the UN website for the period from 2000 to 2009, cleaned, and converted into XML format by specialists from the German center DFKI.

What you can do with it

  • Training and fine-tuning machine translation models (including pairs with Russian)
  • Aligning multilingual embeddings and cross-lingual search
  • Testing translation quality on official and legal vocabulary
Tasks
Translation
Languages
Arabic, German, English, Spanish, French, Russian, Chinese
Size
100M–1B rows
Downloads this month
5.7K
Likes
15

How to load it

from datasets import load_dataset

ds = load_dataset("Helsinki-NLP/multiun")