Stubbi
Rankings
Benchmark

ZeroBench

Can a multimodal model combine several observations from a single image to answer a question?

Explore the original evidence

Explore the original evaluation, its methods, and reported findings.

Open eval

Coverage

The benchmark contains 100 main questions and 334 subquestions. The subquestions isolate steps needed for the full answer. Jonathan Roberts and collaborators released v1 in February 2025, revised it in March and clarified wording in v3 in December 2025.

Methodology

Single-attempt accuracy, success at least once in five attempts and success on all five attempts distinguish occasional solutions from repeatable ones. The leaderboard identifies tool-assisted runs and separates project evaluations from external reports. An August 2026 grading correction kept the questions unchanged. Compare runs using the same tools, question set and grading protocol.

People & organizations

1

Sources

Original paperKimi K3.1 Pro evaluation settingsHacker News discovery