Skip to content

Dataset · HuggingFaceTB

DCLM educational web texts for training small LLMs

HuggingFaceTB/dclm-edu

An educational quality-filtered corpus of English web pages from Hugging Face.

What's inside

An English version of the DCLM corpus filtered by the FineWeb-Edu quality classifier. Each web page is scored on a 0 to 5 scale, with entries scoring above 2 included in the final set (size ranging from 1 billion to 10 billion records). This data was used to train the compact SmolLM2-135M and SmolLM2-360M models.

What you can do with it

  • Pre-training compact language models (SLMs)
  • Selecting clean educational data for LLM fine-tuning
  • Studying the impact of data filtering on small model quality
Languages
English
Size
1B–10B rows
Downloads this month
25.3K
Likes
47

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceTB/dclm-edu")