Skip to content

Dataset · HuggingFaceFW

FineWeb-Edu educational web texts

HuggingFaceFW/fineweb-edu

1.3T tokens of filtered educational web pages in English for LLM training.

What's inside

The dataset contains 1.3 trillion tokens of educational web pages filtered from FineWeb. Filtering utilized an educational quality classifier trained on Llama3-70B-Instruct annotations. A FineWeb-Edu-score-2 version with 5.4 trillion tokens is also available; the card describes the 1.3 trillion version.

What you can do with it

  • pre-training base language models
  • continued pre-training on educational texts
  • evaluating the quality of educational data filtering
Tasks
Text generation
Languages
English
Size
1B–10B rows
Downloads this month
459.9K
Likes
1.5K

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceFW/fineweb-edu")