Benchmark
ViBench 2.0
Test whether a coding agent can keep a web application working as new feature requests arrive.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
Replit and collaborator Georgian developed ViBench. The 2.0 release covers 17 products, 241 feature requests and 544 test plans. Preeya Kirani, Daniel Furman, Peter Zhong and Michele Catasta wrote the release report.
Methodology
The builder starts from an empty project and sees requests one at a time. After the final request, a grading agent uses a browser to test the finished product. Fireworks ran four builds per product and model. A build passes only when every plan succeeds. The separate plan pass rate can reveal partial progress.