Dataset · nuprl
22-language code generation benchmark
nuprl/MultiPL-E
An evaluation set for LLM code generation consisting of HumanEval and MBPP translated into 22 programming languages.
What's inside
The dataset contains text-based coding tasks in 22 programming languages. It was assembled from the OpenAI HumanEval and MBPP benchmarks using small compiler-translators. Configurations are named SRCDATA-LANG, allowing selection of the source benchmark and target language.
What you can do with it
- Evaluate LLMs and coding models on HumanEval/MBPP across different languages
- Compare code generation quality between programming languages
- Test model transfer to low-resource languages from the set
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 30.2K
- Likes
- 71
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("nuprl/MultiPL-E")