Benchmark
Spider 2.0
Evaluate agents querying and transforming data in complex enterprise databases.
Published leaderboard
Explore scores and evaluation settings on the publisher’s leaderboard.
Open leaderboardCoverage
Separate Snow, Lite, and DBT settings: Snow and Lite each contain 547 tasks; DBT contains 68 repository-level tasks.
Methodology
Solve the chosen SQL or data-transformation workflow using its database metadata, documentation, and environment. Evaluate the resulting outputs with the corresponding official task suite.