Dataset · google-research-datasets
MBPP Python Tasks Benchmark
google-research-datasets/mbpp
About 1,000 Python tasks with descriptions, solutions, and tests for code generation evaluation.
What's inside
The Mostly Basic Python Problems benchmark contains about 1,000 crowdsourced tasks for beginners: programming basics, standard library, and more. Each task includes a description, a reference solution, and three automatic tests; part of the data was manually verified by the authors. The language is English.
What you can do with it
- Evaluating LLMs on Python code generation
- Testing code models via pass@k
- Comparing SFT models on the benchmark
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 288.7K
- Likes
- 279
Models trained on it
- mradermacher/Next-3-Pro-i1-GGUF 4.3K
- tmickleydoyle/ghost-typerText generation 1.1K
- mradermacher/Next-3-Pro-GGUF 846
- mradermacher/DeepSeek-R1-Codegeneration-COT-GGUF 796
- mradermacher/Qwen2.5-Math-Coder-Distill-Phi-2-4.4K-MixMathCode-GGUF 639
- raghu1155/DeepSeek-R1-Codegeneration-COTText generation 528
- thelamapi/next3-pro-GGUFImage text → text 359
- sergiopaniego/Qwen3-1.7B-Distillation-CodingText generation 320
How to load it
from datasets import load_dataset
ds = load_dataset("google-research-datasets/mbpp")