Cleaned CommonCrawl world news archive
A massive multilingual corpus of 600 million news articles from 2016 to June 2024.
- Text classification
- Question answering
- Text generation
- Multilingual
- Afrikaans
- Amharic
- +9
Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.
Answered by Gemini 4 · the chat is kept only in this tab
A massive multilingual corpus of 600 million news articles from 2016 to June 2024.
10k to 100k texts of US Congress and California state bills with annotations for summarization tasks.
arXiv papers and abstracts for training long-document summarization in English.
A large cleaned text corpus of 100 million to 1 billion records for training industry LLMs.
52k English instruction-response pairs from GPT-4 based on Alpaca prompts for fine-tuning LLMs.
Under 1,000 Kubernetes cluster incident scenarios for evaluating SRE agents.
Text and tabular tasks for training and evaluating Chinese language understanding models.
262K synthetic prompt-response pairs: harmful and similar harmless prompts for safety training.
Binarized preferences for DPO based on Capybara: multi-turn dialogues for training chat models.
A benchmark of over 24,000 school exam questions across 16 languages for model evaluation.
A collection of 680 questions with answers and site navigation paths for testing AI agents.
1K reasoning examples in Spanish, French, Italian, and German translated from English.
Filtered code corpus of ~240B tokens across 11 programming languages from The Stack v3.
112 billion tokens of agentic coding trajectories across 122k pull requests and 16 programming languages.
A large set of synthetic personas based on real South Korean demographic distributions.
229,122 Australian legal documents in English: legislation and judicial acts.
A corpus of 6.7M US court cases for training and analyzing legal texts.
A large six-language multilingual corpus for training machine translation systems, containing 100M to 1B rows.
Cartoons and captions from the New Yorker contest for evaluating humor understanding in models.
3,008 short stories generated by Claude Opus from Reddit prompts, featuring titles and genres.
11 datasets for fact-checking, QA, entity linking, and dialogue on a single Wikipedia dump.
English interlaced text and image dataset from ArXiv scientific papers for pre-training.
A massive multilingual corpus of European Parliament speech translations for machine translation.
A simplified format of the popular benchmark with programming tasks comprising up to 10,000 examples.