Skip to content

Dataset · openbmb

Cleaned 240B-token source code corpus

openbmb/UltraData-Stack-v3-L2

Filtered code corpus of ~240B tokens across 11 programming languages from The Stack v3.

What's inside

This dataset is a curated source code corpus of roughly 240 billion tokens (100M to 1B records in metadata) spanning 11 programming languages from The Stack v3. Data was cleaned and granularly filtered using the UltraData-Code-L2-Classifier. Metadata lists English and Chinese languages alongside text and tabular modalities.

What you can do with it

  • Pre-training or continual pre-training of code-generation LLMs
  • Training specialized coding assistant models
  • Researching filtering and quality-assessment methods for code corpora
Tasks
Text generation
Languages
English, Chinese
Size
100M–1B rows
Downloads this month
7.4K
Likes
37

How to load it

from datasets import load_dataset

ds = load_dataset("openbmb/UltraData-Stack-v3-L2")