Lean AI formalization leaderboard
Track accepted AI-assisted solutions to formal mathematics problems in Lean.
Overview
lean-eval is a benchmark and public leaderboard for formal mathematics in the Lean proof assistant, run under the leanprover GitHub organisation and the Lean FRO. It collects hard, mostly already-solved mathematical results that can be stated with Mathlib definitions, and tracks which ones external participants have proved formally with AI assistance. A problem counts as solved only when a machine checker accepts the submitted proof.
What a task looks like
Each problem is a trusted Lean statement with one or more declarations ('holes') that the solver must fill in a Submission.lean file, without editing the trusted files; Mathlib may be used. Problems range from a starter '2 + 2 = 4' to formal statements of published research, such as the Erdős–Faber–Lovász conjecture result from the Annals of Mathematics.
How it’s scored
Acceptance is decided entirely by the comparator tool, which checks that the submission proves exactly the trusted statement; readability and style are not judged. The leaderboard ranks submitters' models by how many main benchmark problems have accepted solutions, excluding internal test problems.
- Size
- The catalog changes over time. On 2026-11-05 the problem index listed 312 entries (130 active, 179 archived, 3 draft), 310 in formalization evaluation and 2 in software verification. The frozen LeanEval v1 set has 128 problems (118 at publication plus 10 added on 2026-08-21). The legacy board lists 304 problems.
- Versions
- Named problem sets: LeanEval v1 was frozen and published on 2026-08-20; LeanEval v2 is an unfrozen draft with 2 problems. Problems carry statement revisions. Submission moved from GitHub issue intake (now the legacy route and legacy board) to an authenticated web service, which became primary on 2026-09-02.
- Access
- Public: problem statements and tooling are on GitHub under Apache-2.0, and results are stored in a public submissions repository. Accepted proofs may be released publicly under Apache-2.0 two calendar months after acceptance or kept private.
- Links
- LeaderboardCodeCode
Summarized by Stubbi from the public sources listed below. Stubbi has not run this benchmark.
Published leaderboard
Explore scores and evaluation settings on the publisher’s leaderboard.
Open leaderboardCoverage
A changing problem catalog with externally submitted proof artifacts and statement revisions.
Methodology
Trusted problem statements define comparator checks. Accepted submissions count as solved; public standings preserve problem scope and submitted model identity.