Skip to content

Dataset · datacurve

DeepSWE: Coding Agents Benchmark

datacurve/deep-swe

113 original long-horizon software development tasks in TypeScript, Go, Python, JavaScript, and Rust.

What's inside

A benchmark designed to evaluate frontier agents on original long-horizon tasks from active open-source repositories. It includes 113 tasks in TypeScript, Go, Python, JavaScript, and Rust featuring isolated environments and programmatic verifiers. The tasks use the Harbor task format with metadata covering the repository, commit, language, and resource limits.

What you can do with it

  • Evaluating coding agents on real-world repositories
  • Comparing models in long-horizon SWE
  • Testing agent frameworks
Languages
English
Size
up to 1K rows
Downloads this month
15.6K
Likes
100
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("datacurve/deep-swe")