Dataset · NTU-NLP-sg
xCodeEval code generation benchmark
NTU-NLP-sg/xCodeEval
A large multilingual benchmark for evaluating code LLMs based on program execution rather than text matching.
What's inside
xCodeEval is a benchmark for code tasks including generation, translation, repair, search, and classification. It covers multiple programming languages and evaluates solutions using execution-based metrics instead of lexical overlap alone. The size ranges from 1 to 10 million examples, with code and English data.
What you can do with it
- evaluate LLMs on code generation using execution-based metrics
- test multilingual code translation
- benchmark the retrieval of relevant code snippets
- Tasks
- Translation, Token classification, Text retrieval, Text generation, Text classification, Embeddings
- Languages
- Code, English
- Size
- 1M–10M rows
- Downloads this month
- 41.7K
- Likes
- 85
How to load it
from datasets import load_dataset
ds = load_dataset("NTU-NLP-sg/xCodeEval")