APEX-Accounting
Evaluate whether an agent can close the books for a simulated company using accounting software, spreadsheets and supporting documents.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
Mercor and Ramp built the benchmark with accountants and bookkeepers who created the tasks, reference work and grading criteria. The July launch describes 160 held-out tasks across 10 company worlds and a separate public sample of 10 tasks.
Methodology
The live leaderboard lists 100 tasks across 10 held-out worlds. Agents work in Mercor’s Loop Harness. A model judge checks each rubric criterion separately, allowing partial credit for a reconciliation or journal entry. Mercor gives different task counts for the launch and the live leaderboard. The overlap between those sets is unspecified.