Skip to content

Dataset · princeton-nlp

300 GitHub Tasks for Code Evaluation

princeton-nlp/SWE-bench_Lite

300 Python repository issue-PR pairs for evaluating automated GitHub issue resolution.

What's inside

SWE-bench Lite is a subset of SWE-bench containing 300 test issue and pull request pairs from 11 popular Python repositories. Evaluation is performed by checking unit tests where the benchmark behavior is post-PR. The dataset was released as part of the SWE-bench paper.

What you can do with it

  • Evaluating automated GitHub issue resolution systems
  • Benchmarking LLM agents for code patches
  • Testing code models on real Python tasks
Size
up to 1K rows
Downloads this month
54K
Likes
77

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("princeton-nlp/SWE-bench_Lite")