Skip to content

Dataset · nuprl

22-language code generation benchmark

nuprl/MultiPL-E

An evaluation set for LLM code generation consisting of HumanEval and MBPP translated into 22 programming languages.

What's inside

The dataset contains text-based coding tasks in 22 programming languages. It was assembled from the OpenAI HumanEval and MBPP benchmarks using small compiler-translators. Configurations are named SRCDATA-LANG, allowing selection of the source benchmark and target language.

What you can do with it

  • Evaluate LLMs and coding models on HumanEval/MBPP across different languages
  • Compare code generation quality between programming languages
  • Test model transfer to low-resource languages from the set
Languages
English
Size
10K–100K rows
Downloads this month
30.2K
Likes
71

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nuprl/MultiPL-E")