Dataset · Salesforce
WikiText: Wikipedia texts for language modeling
Salesforce/wikitext
Corpus of verified English Wikipedia articles for training and evaluating language models.
What's inside
The dataset contains over 100M tokens from featured and good English Wikipedia articles. These are English texts collected from Wikipedia and available under the CC BY-SA license. The WikiText-2 and WikiText-103 versions are noticeably larger than the preprocessed Penn Treebank. The format is text data for text-generation and fill-mask tasks.
What you can do with it
- Train a GPT-like model for language modeling
- Test English fill-mask
- Evaluate language model perplexity
- Tasks
- Text generation, Fill mask
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 2.5M
- Likes
- 870
Models trained on it
- nicklas373/Qwen3.5-9B-AWQImage text → text 14.1K
- nicklas373/Huihui-gemma-4-12B-it-abliterated-AWQImage text → text 4.9K
- bayliner1980/Qwen3.5-0.8B-MoE-A0.6B-GGUFText generation 1.8K
- mradermacher/Xyntetik-Kvist-14B-i1-GGUF 1.2K
- TahiraAbbas/gpt2-wikitextText generation 1K
- anwgpt/anwgpt4-baseText generation 845
- bayliner1980/Qwen3.5-0.8B-MoE-A0.6BText generation 710
- Hooshaai/qwen-2.5-0.5b-preconditioned-gdn-attention-wikitext2Text generation 645
How to load it
from datasets import load_dataset
ds = load_dataset("Salesforce/wikitext")