Benchmark
APEX-1
Evaluate a model’s ability to produce professional work from a brief and source documents in banking, consulting, law and primary care.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
Practitioners in four occupations wrote the cases and the criteria for grading them. The extended edition has 100 held-out tasks per occupation, up from 50 in the original. Bertie Vidgen and 19 coauthors document the design.
Methodology
Mercor collects eight single-turn responses per task. A language-model judge checks each response against the expert rubric. Mean score measures the share of criteria satisfied. The 100 public examples are separate from the 400 held-out tasks used for the extended leaderboard.