Skip to content

Dataset · openbmb

GitHub Code with L0–L3 Cleaning Levels

openbmb/UltraData-Code

GitHub code with L0–L3 cleaning levels for training language models.

What's inside

The dataset contains text and tabular code data in English and Chinese. It is an implementation of the tiered UltraData L0–L4 framework, covering code states L0–L3, with each level corresponding to a separate processing stage. The pipeline starts with approximately 192 million public GitHub repositories at level L0, and metadata volume ranges from 100 million to 1 billion records.

What you can do with it

  • Pretraining code LLMs
  • Evaluating code filtering pipelines
  • Studying tiered data management
Tasks
Text generation
Languages
English, Chinese
Size
100M–1B rows
Downloads this month
38.9K
Likes
216

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openbmb/UltraData-Code")