Skip to content

Dataset · NTU-NLP-sg

xCodeEval code generation benchmark

NTU-NLP-sg/xCodeEval

A large multilingual benchmark for evaluating code LLMs based on program execution rather than text matching.

What's inside

xCodeEval is a benchmark for code tasks including generation, translation, repair, search, and classification. It covers multiple programming languages and evaluates solutions using execution-based metrics instead of lexical overlap alone. The size ranges from 1 to 10 million examples, with code and English data.

What you can do with it

  • evaluate LLMs on code generation using execution-based metrics
  • test multilingual code translation
  • benchmark the retrieval of relevant code snippets
Tasks
Translation, Token classification, Text retrieval, Text generation, Text classification, Embeddings
Languages
Code, English
Size
1M–10M rows
Downloads this month
41.7K
Likes
85

How to load it

from datasets import load_dataset

ds = load_dataset("NTU-NLP-sg/xCodeEval")