Skip to content

Dataset · HuggingFaceFW

FineWeb-Edu educational web texts (score threshold 2)

HuggingFaceFW/fineweb-edu-score-2

Massive English web corpus with an educational filter for language model pre-training.

What's inside

Features 5.4 trillion tokens of educational web pages filtered from FineWeb. This version has a lower educational threshold of 2, offering more documents but lower quality than the 1.3T token version. English language data categorized under text-generation.

What you can do with it

  • Pre-training base language models
  • Continued pre-training on educational texts
  • Experiments with data filtering and web data mixing
Tasks
Text generation
Languages
English
Size
10B–100B rows
Downloads this month
32.3K
Likes
90

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceFW/fineweb-edu-score-2")