Dataset · wikimedia
Cleaned Wikisource texts in many languages
wikimedia/wikisource
A corpus of cleaned full Wikisource texts in dozens of languages for training and evaluating language models.
What's inside
The dataset is collected from Wikisource dumps (dumps.wikimedia.org) and contains one subset per language with a single train split. Each example features the full text of a work, cleaned of markdown and extra sections like links. Many languages are available, including Arabic, Assamese, Azerbaijani, Banjarese, Belarusian, and Bulgarian.
What you can do with it
- Fine-tuning a multilingual LLM for text generation
- Evaluating fill-mask on texts in different languages
- Pre-training a compact language model on literary texts
- Tasks
- Text generation, Fill mask
- Languages
- Arabic, Assamese, Azerbaijani, Balinese, Belarusian, Bulgarian, Bangla, Breton and 4 more
- Size
- 1M–10M rows
- Downloads this month
- 13.2K
- Likes
- 88
Models trained on it
- msmth/Source-1Text classification 38
- ThongCoder/vi-distilgpt2Text generation 9
- dwidlee/toyGPTText generation 0
- konko/bengali-bpe-tokenizerToken classification 0
- Lambent/RWKV7-1.5B-midtrain50-docs-lora 0
- Lambent/RWKV7-2.9B-midtrain50-docs-lora 0
- Lambent/RWKV7-7.2B-midtrain50-docs-lora 0
- Lambent/RWKV7-13.3B-midtrain50-docs-lora 0
How to load it
from datasets import load_dataset
ds = load_dataset("wikimedia/wikisource")