Skip to content

Dataset · openbmb

English web corpus Ultra-FineWeb-L1

openbmb/Ultra-FineWeb-L1

Large English web corpus from Common Crawl, filtered L1 layer for training language models.

What's inside

Ultra-FineWeb-L1 is a large-scale English corpus collected from Common Crawl snapshots. Within the UltraData L0–L4 tiers, it is a filtered L1 layer for general web data and the basis for further L2/L3. Processing is built on the FineWeb pipeline; metadata volume is in the 1B–10B range.

What you can do with it

  • Pre-training or fine-tuning English LLMs
  • Web data filtering experiments
  • Comparing corpus quality for pretraining
Tasks
Text generation
Languages
English
Size
1B–10B rows
Downloads this month
6.4K
Likes
203

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openbmb/Ultra-FineWeb-L1")