Dataset · HuggingFaceTB
DCLM educational web texts for training small LLMs
HuggingFaceTB/dclm-edu
An educational quality-filtered corpus of English web pages from Hugging Face.
What's inside
An English version of the DCLM corpus filtered by the FineWeb-Edu quality classifier. Each web page is scored on a 0 to 5 scale, with entries scoring above 2 included in the final set (size ranging from 1 billion to 10 billion records). This data was used to train the compact SmolLM2-135M and SmolLM2-360M models.
What you can do with it
- Pre-training compact language models (SLMs)
- Selecting clean educational data for LLM fine-tuning
- Studying the impact of data filtering on small model quality
- Languages
- English
- Size
- 1B–10B rows
- Downloads this month
- 25.3K
- Likes
- 47
Models trained on it
- Meridian21Lab/concept-memory-140m-11.5btText generation 701
- keeeeenw/MicroLlama2-baseText generation 586
- Meridian-MRM/Zedev-134mText generation 564
- Rinstitute/Rina-Juvenile-Exp-2B-GGUFText generation 547
- SupraLabs/MicroSupra-10kText generation 219
- wzebrowski/OpenSML-JSONLite-175MText generation 210
- sz14/cRia-LM-75MText generation 193
- OpenSML/OpenSML-JSONLite-175MText generation 184
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceTB/dclm-edu")