Open government data corpus with 380 billion tokens
Massive text and table corpus from open government archives of the USA, France, and EU for language model pre-training.
- Text generation
- English
- French
- Bulgarian
- +9
Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.
Answered by Gemini 4 · the chat is kept only in this tab
Massive text and table corpus from open government archives of the USA, France, and EU for language model pre-training.
Corpus of French public domain newspapers and periodicals for language models.
Open collection of real developer dialogs with AI agents, including tool calls and command outputs.
1,781 agent pipeline execution logs for performance analysis and inference optimization.
Tabular and text data on pushes, pull requests, releases, and discussions from public repositories.
A large-scale text dataset with difficulty grading for SFT and enhancing model reasoning skills.
173 million eukaryotic and prokaryotic DNA and RNA sequences for pretraining base genomic models.
26,750 open prompts in over 100 languages for evaluating text generation quality.
Over 700,000 synthetic Chinese examples based on SmolTalk for SFT.
Nearly deduplicated version of The Stack v2—a large code corpus for LLM training, grouped by repositories.
Dataset for evaluating medical language models across 35 categories.
110k Chinese examples with answers and R1 reasoning traces for SFT.
50k unfiltered instruction-response pairs in English for training language models.
582 trajectories and 3,956 rows for distilling agentic coding and tool use.
Archive of US public company annual reports spanning three decades in parquet format for financial LLMs.
One million Rust source code files from GitHub for training and fine-tuning compact language models.
European newspaper texts from the 18th to early 20th century for training and research.
10,331 community-rated prompts for training and evaluating prompt ranking.
Deduplicated SFT dataset of 10M to 100M rows for training reasoning, programming, math, and cybersecurity.
340 legal exams from Switzerland, the EU, and international law for evaluating reasoning models.
Over 100,000 records of AI agent shopping sessions for text-generation and RL tasks.
Over 150 million bibliographic records from the Open Library project for NLP, search, and data extraction tasks.
Professionally translated Wikipedia sentences in 101 languages for evaluating multilingual translation quality.
Large cleaned Uzbek language corpus in Latin and Cyrillic scripts for model pre-training.