Skip to content

Dataset · openbmb

Ultra-FineWeb-L3 cleaned web corpus

openbmb/Ultra-FineWeb-L3

High-quality English and Chinese L3 web texts with enhanced reasoning signals for LLM training.

What's inside

Ultra-FineWeb-L3 is the L3-level cleaning of general high-quality web data within the UltraData multi-level system (L0–L4). The data is sourced from valuable web corpora and transformed into structured, more easily digestible training material with clearer reasoning signals and educational content. Languages are English and Chinese; metadata size is 1B–10B.

What you can do with it

  • Pre-training or continued pretraining of LLMs on the cleaned web corpus
  • Fine-tuning models on English-Chinese educational texts
  • Creating high-quality SFT data with reasoning signals
Tasks
Text generation
Languages
English, Chinese
Size
1B–10B rows
Downloads this month
20.2K
Likes
347

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openbmb/Ultra-FineWeb-L3")