Skip to content

Dataset · codeparrot

115M GitHub code files

codeparrot/github-code-clean

A filtered version of GitHub code containing 115M files across 32 programming languages, totaling nearly 1 TB of text.

What's inside

This is a more filtered version of the codeparrot/github-code dataset. It includes 115M GitHub code files across 32 programming languages and 60 extensions, totaling nearly 1 TB of textual data.

What you can do with it

  • Pre-training code LLMs
  • Training code auto-completion
  • Researching GitHub code filtering
Size
10M–100M rows
Downloads this month
53.4K
Likes
151

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("codeparrot/github-code-clean")