WikiText: Wikipedia texts for language modeling
Corpus of verified English Wikipedia articles for training and evaluating language models.
- Text generation
- Fill mask
- English
Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.
Answered by Gemini 4 · the chat is kept only in this tab
Corpus of verified English Wikipedia articles for training and evaluating language models.
8.5K diverse grade school math problems with step-by-step solutions for reasoning evaluation.
Massive cleaned Common Crawl corpus with en, realnewslike, and multilingual mC4 variants.
Multiple-choice QA benchmark across 57 subjects in English.
7,887 multiple-choice science questions for evaluating question-answering models.
Large-scale multilingual archive of news articles from 2016–2026 with quality filtering and deduplication.
1.3T tokens of filtered educational web pages in English for LLM training.
Large English web corpus broken down by dozens of domains and iteration statistics for training and classification.
12k complex multi-discipline questions for rigorous LLM evaluation.
Top 10% highest-quality and structured web texts from the FineWeb corpus for language model training.
Over 18.5 trillion tokens of cleaned and deduplicated English web texts from CommonCrawl for LLM training.
Uncleaned archive of correspondence from Ubuntu IRC chats for training text generation language models.
Cleaned Wikipedia articles in multiple languages, with one train split per language.
English and German text scripts with sudden emotional shifts for a single character to train expressive TTS models.
Large-scale English corpus of 1.61 million web novel records containing text, tables, and images.
Over 1.1 million hours of web audio in 100+ languages for ASR, TTS, and audio representation learning.
A massive dataset of over 1.1 million hours of audio across 100+ languages for speech tasks.
Synthesized and processed articles with metadata in 5 languages for text generation, translation, and summarization tasks.
A subset of 500 math problems from the MATH benchmark for evaluating step-by-step reasoning.
Synthetic short stories with a limited English vocabulary for training and evaluating language models.
English MINT-1T HTML corpus: part of an open 1 trillion token and 3.4 billion image multimodal dataset.
FineWeb-2 pretraining corpus with 1000+ languages for training language models.
Challenging NLU tasks including textual entailment, QA, and coreference resolution for language model evaluation.
Around 500 verifiable instructions for evaluating how well LLMs follow explicit constraints.