Dataset · xlangai
DS-1000 code generation benchmark
xlangai/DS-1000
A simplified format of the popular benchmark with programming tasks comprising up to 10,000 examples.
What's inside
It contains text-based coding tasks (including the original fill-in-the-middle format). The dataset size ranges from 1 to 10 thousand examples. This is a reformatted and simplified version of the DS-1000 benchmark, associated with the Eval-Arena leaderboard.
What you can do with it
- Evaluating the ability of language models to write and complete code
- Comparing code LLMs using Eval-Arena metrics
- Tasks
- Text generation
- Languages
- Code
- Size
- 1K–10K rows
- Downloads this month
- 7K
- Likes
- 39
How to load it
from datasets import load_dataset
ds = load_dataset("xlangai/DS-1000")