Benchmark
SWE-bench
Evaluate coding agents on resolving real software repository issues.
Published leaderboard
Explore scores and evaluation settings on the publisher’s leaderboard.
Open leaderboardCoverage
The creator's research page links the original benchmark and Verified, Multimodal, and Multilingual variants. A specific variant has not yet been selected for a Stubbi snapshot.
Methodology
Consult the selected variant's official task definitions, evaluation harness, and submission details before comparing systems.