Skip to content

Dataset · singletongue

Cleaned 6-language Wikipedia paragraphs with annotations

singletongue/wikipedia-paragraphs

Large multilingual corpus of Wikipedia paragraphs with internal links, categories, and Wikidata identifiers.

What's inside

The corpus contains from 100 million to 1 billion records in English, German, Spanish, French, Italian, and Arabic, broken down by language configurations (for example, the fresh enwiki dump from November 2026). Each record is a cleaned paragraph of Wikipedia text, supplemented with information about internal wikilinks, categories, templates, and Wikidata QID mapping. The data format combines text and tabular metadata.

What you can do with it

  • Training and fine-tuning Entity Linking models with Wikidata
  • Building multilingual information retrieval systems and RAG knowledge bases
  • Pre-training and evaluation of language models on clean encyclopedic texts
Languages
Arabic, German, English, Spanish, French, Italian, Japanese, Korean and 3 more
Size
100M–1B rows
Downloads this month
22.4K
Likes
2

How to load it

from datasets import load_dataset

ds = load_dataset("singletongue/wikipedia-paragraphs")