Benchmark
API-Bank
Can a language model plan a sequence of tool calls, find the right API and call it with the information a user supplied?
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
The evaluation includes 73 executable tools and 314 annotated conversations containing 753 calls.
Methodology
The tasks exercise tool selection, retrieval and multi-step planning inside a runnable environment. The authors provide a separate collection of 1,888 training dialogues. The evaluation measures tool use in this controlled environment. The larger training collection draws on 2,138 APIs.