Benchmark
AI2 Diagrams (AI2D)
Evaluate diagram structure understanding and question answering about scientific illustrations.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
The Allen Institute dataset registry lists 4,817 illustrative diagrams. The original paper studies diagram parsing and question answering.
Methodology
Introduced by Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi and Ali Farhadi. Diagram Parse Graphs describe visual elements and their relationships; question answering tests interpretation of the diagrams. This entry covers the original 2016 dataset; later reannotations and evaluator subsets use different protocols.