Dataset · bigcode
The Stack v2 code dataset with IDs for training
bigcode/the-stack-v2-train-full-ids
Cleaned code across 600+ programming languages, grouped by repository, for training LLMs.
What's inside
The Stack v2 train-full-ids version based on the deduplicated The Stack v2, further filtered by heuristics. The data covers 600+ programming languages and is grouped by repository. In the metadata, the size is specified as 10-100 million examples; this is a code corpus for training generative models.
What you can do with it
- Pretrain code LLMs
- Fine-tune code auto-completion models
- Research code quality across languages
- Tasks
- Text generation
- Languages
- Code
- Size
- 10M–100M rows
- Downloads this month
- 1.2K
- Likes
- 73
- Access
- on request on Hugging Face
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("bigcode/the-stack-v2-train-full-ids")