Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe
On the map: LLM agents and tool use → Tool learning and function calling
Abstract (arXiv)
Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across live government APIs. However, open-source models consistently underperform in this multi-step setting, and no existing benchmark measures the gap. We introduce the Korean Open Public API Benchmark (KOPA-Bench), comprising 145 real-world tasks. To close this gap, we present EDGE, an Execution-grounded Dynamic Graph for tool-calling data synthEsis driven by live execution. EDGE builds a graph of how each tool's output can feed another's input, keeps only the links that succeed when actually called against the live APIs, and traverses these verified links to synthesize executable multi-step trajectories. Fine-tuned via GRPO on the resulting dataset, our 9B model nearly matches the untuned 27B model from the same family, improving substantially not only on KOPA-Bench but also on the BFCL benchmark.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 12 of its 34 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
12 of 34 references.
Closest ancestors · 5
Same direction, newest first
- 2025Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling30 Sep
- 2024FunctionChat-Bench: Comprehensive Evaluation of Language Models' Generative Capabilities in Korean Tool-use Dialogs21 Nov
- 2024Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning16 Oct
- 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets26 Jun
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
Other references · 7
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 152
- 2026DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Taskscited by 49
- 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs
- 2025Live API-Bench: 2500+ Live APIs for Testing Multi-Step Tool Calling
- 2025In-N-Out: A Parameter-Level API Graph Dataset for Tool Agents
- 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls
- 2026ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox
- 2026Agent-Diff: Benchmarking LLM Agents on Enterprise API Tasks via Code Execution with State-Diff-Based Evaluation
- 2026LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
- 2026E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios