Benchmark
QA Bench
How many real bugs can a QA agent find in a running web app, and how often are its reports accurate?
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
Momentic covers 20 product areas across Gitea, Metabase, Twenty and Saleor. The reference set contains 1,396 reviewed bugs.
Methodology
Agents inspect one area for 30 or 60 minutes. A model judge matches their reports to labels to measure precision and recall. Momentic also builds a tested agent. Its labels draw on bugs proposed by the evaluated agents, and the product areas have different numbers of labeled bugs.