Dataset · HuggingFaceFW
FineWeb-Edu educational web texts
HuggingFaceFW/fineweb-edu
1.3T tokens of filtered educational web pages in English for LLM training.
What's inside
The dataset contains 1.3 trillion tokens of educational web pages filtered from FineWeb. Filtering utilized an educational quality classifier trained on Llama3-70B-Instruct annotations. A FineWeb-Edu-score-2 version with 5.4 trillion tokens is also available; the card describes the 1.3 trillion version.
What you can do with it
- pre-training base language models
- continued pre-training on educational texts
- evaluating the quality of educational data filtering
- Tasks
- Text generation
- Languages
- English
- Size
- 1B–10B rows
- Downloads this month
- 459.9K
- Likes
- 1.5K
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb-edu")