Dataset · ammarnasr
Cleaned Rust source code from TheStack corpus
ammarnasr/the-stack-rust-clean
One million Rust source code files from GitHub for training and fine-tuning compact language models.
What's inside
Contains Rust programming language source code files selected from the large TheStack corpus (GitHub data). The set includes a total of 1,000,000 examples: 900,000 for training, 50,000 for validation, and 50,000 for testing. The dataset is pre-cleaned and presented in tabular and text formats.
What you can do with it
- Training or fine-tuning compact LLMs for Rust code generation and auto-completion
- Creating a specialized LoRA for programming assistants
- Evaluating Rust code generation quality on a test split of 50 thousand files
- Tasks
- Text generation
- Languages
- Code
- Size
- 100K–1M rows
- Downloads this month
- 2.4K
- Likes
- 26
Models trained on it
- mratsim/MiniMax-M2.5-FP8-INT4-AWQText generation 384
- or4cl3ai/Aiden_t5Text generation 181
- mratsim/MiniMax-M2.5-BF16-INT4-AWQText generation 82
- mratsim/MiniMax-M2.1-FP8-INT4-AWQText generation 74
- mratsim/MiniMax-M2.1-BF16-INT4-AWQText generation 25
- lovethayo/MiniMax-M2.5-FP8-INT4-AWQText generation 20
- ammarnasr/codegen-350M-mono-rustText generation 8
- Superuser666-Sigil/Llama-3.1-8B-Instruct-Rust-QLoraText generation 0
How to load it
from datasets import load_dataset
ds = load_dataset("ammarnasr/the-stack-rust-clean")