Dataset · openbmb
Ultra-FineWeb-L3 cleaned web corpus
openbmb/Ultra-FineWeb-L3
High-quality English and Chinese L3 web texts with enhanced reasoning signals for LLM training.
What's inside
Ultra-FineWeb-L3 is the L3-level cleaning of general high-quality web data within the UltraData multi-level system (L0–L4). The data is sourced from valuable web corpora and transformed into structured, more easily digestible training material with clearer reasoning signals and educational content. Languages are English and Chinese; metadata size is 1B–10B.
What you can do with it
- Pre-training or continued pretraining of LLMs on the cleaned web corpus
- Fine-tuning models on English-Chinese educational texts
- Creating high-quality SFT data with reasoning signals
- Tasks
- Text generation
- Languages
- English, Chinese
- Size
- 1B–10B rows
- Downloads this month
- 20.2K
- Likes
- 347
Models trained on it
- openbmb/MiniCPM5-2BText generation 923.4K
- openbmb/MiniCPM5-2B-GGUFText generation 678.9K
- openbmb/MiniCPM5-2B-DSparkText generation 306.8K
- openbmb/MiniCPM5-1BText generation 227.2K
- openbmb/MiniCPM5-2B-MLXText generation 137.4K
- Vxtzq/Crowd-v1Text generation 94.1K
- openbmb/MiniCPM5-1B-GGUFText generation 35.9K
- openbmb/MiniCPM5-1B-MLXText generation 29.8K
How to load it
from datasets import load_dataset
ds = load_dataset("openbmb/Ultra-FineWeb-L3")