WikiText: Wikipedia texts for language modeling
Corpus of verified English Wikipedia articles for training and evaluating language models.
- Text generation
- Fill mask
- English
Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.
Answered by Gemini 4 · the chat is kept only in this tab
Corpus of verified English Wikipedia articles for training and evaluating language models.
Large multimodal corpus of human hand manipulation videos with motion annotations and descriptions for robot learning.
8.5K diverse grade school math problems with step-by-step solutions for reasoning evaluation.
1K examples of synthetic reasoning across math, science, code, and puzzles.
Massive cleaned Common Crawl corpus with en, realnewslike, and multilingual mC4 variants.
Multiple-choice QA benchmark across 57 subjects in English.
Digitized Arabic Islamic books from the Prophet's Mosque Library for OCR and language models.
7,887 multiple-choice science questions for evaluating question-answering models.
1 to 10 thousand JSON records with scores of various models from leaderboards, research papers, and local runs.
Collection of tasks and data for training and evaluating English text understanding systems.
1.3T tokens of filtered educational web pages in English for LLM training.
Large-scale multilingual archive of news articles from 2016–2026 with quality filtering and deduplication.
Large English web corpus broken down by dozens of domains and iteration statistics for training and classification.
Generated images from complex prompts to check composition and logical relations in text-to-image models.
Multimodal web interaction recordings formatted as dialogs for training web agents.
12k complex multi-discipline questions for rigorous LLM evaluation.
Top 10% highest-quality and structured web texts from the FineWeb corpus for language model training.
Over 18.5 trillion tokens of cleaned and deduplicated English web texts from CommonCrawl for LLM training.
Millions of records with language model rating history from the popular Arena benchmark.
Uncleaned archive of correspondence from Ubuntu IRC chats for training text generation language models.
Cleaned corpus of 31.8 million news articles for 2016-2026 for forecasting systems.
Cleaned Wikipedia articles in multiple languages, with one train split per language.
10,000 to 100,000 examples featuring text, images, and 3D code for evaluating and training 3D generation models.
220 expert tasks across 44 professions with prompts and files for evaluating AI on economically significant work.