Dataset · singletongue
Cleaned 6-language Wikipedia paragraphs with annotations
singletongue/wikipedia-paragraphs
Large multilingual corpus of Wikipedia paragraphs with internal links, categories, and Wikidata identifiers.
What's inside
The corpus contains from 100 million to 1 billion records in English, German, Spanish, French, Italian, and Arabic, broken down by language configurations (for example, the fresh enwiki dump from November 2026). Each record is a cleaned paragraph of Wikipedia text, supplemented with information about internal wikilinks, categories, templates, and Wikidata QID mapping. The data format combines text and tabular metadata.
What you can do with it
- Training and fine-tuning Entity Linking models with Wikidata
- Building multilingual information retrieval systems and RAG knowledge bases
- Pre-training and evaluation of language models on clean encyclopedic texts
- Languages
- Arabic, German, English, Spanish, French, Italian, Japanese, Korean and 3 more
- Size
- 100M–1B rows
- Downloads this month
- 22.4K
- Likes
- 2
How to load it
from datasets import load_dataset
ds = load_dataset("singletongue/wikipedia-paragraphs")