Dataset · UnipatAI
Coding agent benchmark on repository upgrades
UnipatAI/RoadmapBench
115 complex software development tasks from real open-source project updates across five programming languages.
What's inside
The benchmark includes 115 multi-step English tasks sourced from 17 open-source repositories in Python, TypeScript, Go, Rust, and C++. In each task, the agent must implement multiple interrelated features corresponding to a real project version update. In addition to text, image modality is noted in the metadata.
What you can do with it
- Evaluating autonomous AI agents for software development
- Testing LLMs on long-term planning of codebase modifications
- Tasks
- Text generation
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 56.9K
- Likes
- 12
How to load it
from datasets import load_dataset
ds = load_dataset("UnipatAI/RoadmapBench")