Skip to content

Dataset · HuggingFaceFW

Cleaned English web corpus

HuggingFaceFW/fineweb

Over 18.5 trillion tokens of cleaned and deduplicated English web texts from CommonCrawl for LLM training.

What's inside

FineWeb is a corpus of English web texts from CommonCrawl totaling over 18.5 trillion tokens (initially 15 trillion). The data is cleaned and deduplicated by a datatrove library pipeline optimized for LLM quality. The metadata indicates tabular and text modalities, a size of 10B-100B, and the en language.

What you can do with it

  • Pretraining LLMs
  • Fine-tuning language models on web texts
  • Testing data cleaning pipelines
Tasks
Text generation
Languages
English
Size
10B–100B rows
Downloads this month
361.5K
Likes
3.8K

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceFW/fineweb")