Stubbi
Rankings
Benchmark

API-Bank

Can a language model plan a sequence of tool calls, find the right API and call it with the information a user supplied?

Explore the original evidence

Explore the original evaluation, its methods, and reported findings.

Open eval

Coverage

The evaluation includes 73 executable tools and 314 annotated conversations containing 753 calls.

Methodology

The tasks exercise tool selection, retrieval and multi-step planning inside a runnable environment. The authors provide a separate collection of 1,888 training dialogues. The evaluation measures tool use in this controlled environment. The larger training collection draws on 2,138 APIs.

People & organizations

9

Sources

Preprint and revision history