Skip to content

Dataset · UnipatAI

Coding agent benchmark on repository upgrades

UnipatAI/RoadmapBench

115 complex software development tasks from real open-source project updates across five programming languages.

What's inside

The benchmark includes 115 multi-step English tasks sourced from 17 open-source repositories in Python, TypeScript, Go, Rust, and C++. In each task, the agent must implement multiple interrelated features corresponding to a real project version update. In addition to text, image modality is noted in the metadata.

What you can do with it

  • Evaluating autonomous AI agents for software development
  • Testing LLMs on long-term planning of codebase modifications
Tasks
Text generation
Languages
English
Size
up to 1K rows
Downloads this month
56.9K
Likes
12

How to load it

from datasets import load_dataset

ds = load_dataset("UnipatAI/RoadmapBench")