Dataset · benchmark-anon-2026
Complex multi-step benchmark for coding agents
benchmark-anon-2026/RoadmapBench
115 real-world version upgrade tasks for open-source projects across 5 programming languages.
What's inside
An English-language benchmark for evaluating AI programming agents on long planning horizons. It contains 115 tasks from 17 open-source repositories in Python, TypeScript, Go, Rust, and C++. Each task is based on a real project version upgrade and requires implementing several interrelated features.
What you can do with it
- Testing autonomous coding agents on complex refactoring and codebase evolution tasks
- Comparing LLM context retention when working with an entire repository
- Tasks
- Text generation
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 28K
- Likes
- 1
How to load it
from datasets import load_dataset
ds = load_dataset("benchmark-anon-2026/RoadmapBench")