Dataset · HuggingFaceFW
Cleaned English web corpus
HuggingFaceFW/fineweb
Over 18.5 trillion tokens of cleaned and deduplicated English web texts from CommonCrawl for LLM training.
What's inside
FineWeb is a corpus of English web texts from CommonCrawl totaling over 18.5 trillion tokens (initially 15 trillion). The data is cleaned and deduplicated by a datatrove library pipeline optimized for LLM quality. The metadata indicates tabular and text modalities, a size of 10B-100B, and the en language.
What you can do with it
- Pretraining LLMs
- Fine-tuning language models on web texts
- Testing data cleaning pipelines
- Tasks
- Text generation
- Languages
- English
- Size
- 10B–100B rows
- Downloads this month
- 361.5K
- Likes
- 3.8K
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb")