Dataset · bigcode
Cleaned code identifiers from The Stack v2
bigcode/the-stack-v2-train-smol-ids
Subsample of deduplicated source code identifiers for training generative models.
What's inside
Contains tabular and text data in programming languages (10 to 100 million records). This is a compact version based on the deduplicated The Stack v2 corpus from the BigCode project, filtered by heuristics and grouped by repository.
What you can do with it
- Pre-train models for code generation and auto-completion
- Collect and filter a custom source code corpus by repository IDs
- Tasks
- Text generation
- Languages
- Code
- Size
- 10M–100M rows
- Downloads this month
- 830
- Likes
- 65
- Access
- on request on Hugging Face
Models trained on it
- mradermacher/Gaperon-1125-8B-GGUF 1.2K
- mradermacher/Gaperon-Young-1125-8B-GGUF 660
- mradermacher/Gaperon-Garlic-1125-8B-GGUF 487
- pfnet/plamo-3-nict-2b-baseText generation 307
- mradermacher/Gaperon-1125-1B-GGUF 255
- pfnet/plamo-3-nict-8b-baseText generation 223
- mradermacher/Gaperon-Young-1125-1B-GGUF 177
- mradermacher/Gaperon-Garlic-1125-1B-GGUF 115
How to load it
from datasets import load_dataset
ds = load_dataset("bigcode/the-stack-v2-train-smol-ids")