Dataset · codeparrot
Cleaned Python Code from GitHub
codeparrot/codeparrot-clean
Deduplicated GitHub Python file corpus for training code generation models.
What's inside
This is a cleaned version of the CodeParrot dataset: Python files from GitHub. Exact duplicates and noise were removed from the source data by limiting line length and alphanumeric character ratio, and removing autogenerated files. Modalities include tabular data and text; size ranges from 1 to 10 million records.
What you can do with it
- Pretraining small code LLMs
- Fine-tuning Python auto-completion models
- Evaluating code cleaning quality
- Size
- 1M–10M rows
- Downloads this month
- 33.1K
- Likes
- 92
Models trained on it
- codeparrot/codeparrot-smallText generation 112.6K
- nkthebass/tinybrainbot-360m-a75m-v5.1-pythonText generation 851
- esraderey/chispa-20m-pythonText generation 760
- vamboai/morena-1.5b-instruct-gguf 562
- ahmetggg/Luck-spark-V-300-MoEText generation 558
- toonist/AnuLM-Base-v2-400MText generation 484
- mradermacher/tinybrainbot-360m-a75m-v5.1-python-GGUF 454
- toonist/AnuLM-Coder-400MText generation 415
How to load it
from datasets import load_dataset
ds = load_dataset("codeparrot/codeparrot-clean")