Dataset · datacurve
DeepSWE: Coding Agents Benchmark
datacurve/deep-swe
113 original long-horizon software development tasks in TypeScript, Go, Python, JavaScript, and Rust.
What's inside
A benchmark designed to evaluate frontier agents on original long-horizon tasks from active open-source repositories. It includes 113 tasks in TypeScript, Go, Python, JavaScript, and Rust featuring isolated environments and programmatic verifiers. The tasks use the Harbor task format with metadata covering the repository, commit, language, and resource limits.
What you can do with it
- Evaluating coding agents on real-world repositories
- Comparing models in long-horizon SWE
- Testing agent frameworks
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 15.6K
- Likes
- 100
- Access
- on request on Hugging Face
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("datacurve/deep-swe")