Dataset · openbmb
Large Filtered En/Zh Web Corpus
openbmb/Ultra-FineWeb
1-10 billion English and Chinese texts after a quality check pipeline.
What's inside
A large English and Chinese text corpus filtered via an efficient quality check pipeline. Assembled from FineWeb and Chinese FineWeb; source data includes IndustryCorpus2, MiChao, WuDao, SkyPile, among others. Metadata indicates a volume of 1 to 10 billion examples.
What you can do with it
- Pretrain language model on en/zh
- Continued pretraining with clean web text
- Evaluate data filtering pipelines
- Tasks
- Text generation
- Languages
- English, Chinese
- Size
- 1B–10B rows
- Downloads this month
- 40.1K
- Likes
- 466
Models trained on it
- openbmb/MiniCPM5-2BText generation 923.4K
- openbmb/MiniCPM5-2B-GGUFText generation 662.9K
- openbmb/MiniCPM5-2B-DSparkText generation 306.8K
- openbmb/MiniCPM5-1BText generation 227.2K
- openbmb/MiniCPM5-2B-MLXText generation 137.4K
- openbmb/MiniCPM5-1B-GGUFText generation 35.9K
- openbmb/MiniCPM5-1B-MLXText generation 29.8K
- openbmb/MiniCPM5-2B-BaseText generation 11K
How to load it
from datasets import load_dataset
ds = load_dataset("openbmb/Ultra-FineWeb")