Benchmark
WebMall
Evaluate comparison shopping across stores, including compatible products and checkout.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
Version 1.0 covers 91 tasks across four simulated stores with 4,421 electronics offers, including substitutes, compatibility, price comparison, cart, and checkout tasks.
Methodology
Exact task completion and macro-averaged precision, recall, and F1 check returned offers or transaction state. Steps, tokens, runtime, and cost provide separate efficiency measures.