Dataset · HuggingFaceFW
FineWeb-Edu educational web texts (score threshold 2)
HuggingFaceFW/fineweb-edu-score-2
Massive English web corpus with an educational filter for language model pre-training.
What's inside
Features 5.4 trillion tokens of educational web pages filtered from FineWeb. This version has a lower educational threshold of 2, offering more documents but lower quality than the 1.3T token version. English language data categorized under text-generation.
What you can do with it
- Pre-training base language models
- Continued pre-training on educational texts
- Experiments with data filtering and web data mixing
- Tasks
- Text generation
- Languages
- English
- Size
- 10B–100B rows
- Downloads this month
- 32.3K
- Likes
- 90
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb-edu-score-2")