Skip to content

Dataset · wikimedia

Cleaned Wikisource texts in many languages

wikimedia/wikisource

A corpus of cleaned full Wikisource texts in dozens of languages for training and evaluating language models.

What's inside

The dataset is collected from Wikisource dumps (dumps.wikimedia.org) and contains one subset per language with a single train split. Each example features the full text of a work, cleaned of markdown and extra sections like links. Many languages are available, including Arabic, Assamese, Azerbaijani, Banjarese, Belarusian, and Bulgarian.

What you can do with it

  • Fine-tuning a multilingual LLM for text generation
  • Evaluating fill-mask on texts in different languages
  • Pre-training a compact language model on literary texts
Tasks
Text generation, Fill mask
Languages
Arabic, Assamese, Azerbaijani, Balinese, Belarusian, Bulgarian, Bangla, Breton and 4 more
Size
1M–10M rows
Downloads this month
13.2K
Likes
88

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("wikimedia/wikisource")