Stubbi
Rankings
Benchmark

AGIEval

Test models on questions from human admission and qualification exams.

Checked Oct 22, 2026Microsoft

Overview

AGIEval evaluates foundation models with questions taken from standardized human exams rather than purpose-built machine-learning datasets. The exams include college entrance tests, law school admission tests, lawyer qualification exams, and math competitions, in English and Chinese. The aim is to gauge general ability on tasks that people actually sit for.

What a task looks like

A task item is an exam question, usually multiple choice, with an optional reading passage, the question, answer options, and a label. The repository's format example is a Chinese math question about the intersection of two sets with four options; a couple of tasks (for example Gaokao-Math-Cloze and MATH) are fill-in-the-blank.

How it’s scored

Answers are checked against the key and reported as accuracy per task plus an average, using the repository's post-processing and evaluation script. No LLM judge is described in the sources.

Released
2023-04
Size
20 tasks in v1.1: 18 multiple-choice and 2 cloze. The sources fetched do not give a total question count in text.
Versions
Paper v1 (13 April 2023) and v2 (18 September 2023). The GitHub data is at v1.1, which added 2023 Gaokao chemistry, biology, and physics questions, fixed annotation problems, and made every multiple-choice task single-answer; the English subset is unchanged from v1.0.
Access
Public data in the GitHub repository, MIT licensed, with the note that data use should follow the licenses of the original source datasets.
Links
PaperCode

Summarized by Stubbi from the public sources listed below. Stubbi has not run this benchmark.

Explore the original evidence

Explore the original evaluation, its methods, and reported findings.

Open eval

Coverage

Twenty tasks from exams such as the SAT, LSAT, Chinese college entrance exams, law exams, and math competitions, in English and Chinese.

Methodology

Score multiple-choice and fill-in answers; report accuracy by exam and overall.

People & organizations

5

Sources

Original paper and author team