Dataset · bigcode
The Stack source code corpus
bigcode/the-stack
A large source code corpus in dozens of programming languages with permissive licenses.
What's inside
This is source code from GitHub: v1.0 featured 30 programming languages and 18 permissive licenses, totaling 3 TB after near-deduplication. In v1.1, weak copyleft licenses MPL/EPL/LGPL are excluded, and the list of permissive licenses is expanded to 193. The data is presented as tabular and text, with code as the language.
What you can do with it
- Train LLMs for code auto-completion
- Research licenses and code filtering
- Fine-tune a code model on permissive code
- Tasks
- Text generation
- Languages
- Code
- Size
- 100M–1B rows
- Downloads this month
- 115.1K
- Likes
- 1.1K
- Access
- on request on Hugging Face
Models trained on it
- ku-nlp/deberta-v3-base-japanese 12.4K
- bigcode/santacoderText generation 7K
- bigcode/gpt_bigcode-santacoderText generation 6.5K
- mradermacher/TildeOpen-30B-Instruct-LV-i1-GGUF 4.8K
- TildeAI/TildeOpen-30bText generation 2.3K
- JetBrains/Mellum-4b-baseText generation 2K
- mradermacher/mpt-7b-8k-i1-GGUF 1.8K
- mradermacher/mpt-7b-i1-GGUF 1.1K
How to load it
from datasets import load_dataset
ds = load_dataset("bigcode/the-stack")