Dataset · openbmb
English web corpus Ultra-FineWeb-L1
openbmb/Ultra-FineWeb-L1
Large English web corpus from Common Crawl, filtered L1 layer for training language models.
What's inside
Ultra-FineWeb-L1 is a large-scale English corpus collected from Common Crawl snapshots. Within the UltraData L0–L4 tiers, it is a filtered L1 layer for general web data and the basis for further L2/L3. Processing is built on the FineWeb pipeline; metadata volume is in the 1B–10B range.
What you can do with it
- Pre-training or fine-tuning English LLMs
- Web data filtering experiments
- Comparing corpus quality for pretraining
- Tasks
- Text generation
- Languages
- English
- Size
- 1B–10B rows
- Downloads this month
- 6.4K
- Likes
- 203
Models trained on it
- VertexResearch/Vertex-0.6-35M-BaseText generation 268
- RikkaBotan/LR-AttnRes-BERT-tinyFill mask 130
- OliverSundaram/MoE-Study-RemasteredText generation 47
- ProCreations/boopit-1-chatText generation 5
- ProCreations/booper-pretrain 0
- ProCreations/booper-chatText generation 0
- Sarthak1134/MAVERIKImage text → video 0
How to load it
from datasets import load_dataset
ds = load_dataset("openbmb/Ultra-FineWeb-L1")