mlsanityData to 14 Oct 2026

On the Robustness of Agentic Function Calling

arXiv 2504.00914 ↗First posted 1 Apr 2025cs.CLcited by 19

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Large Language Models (LLMs) are increasingly acting as autonomous agents, with function calling (FC) capabilities enabling them to invoke specific tools for tasks. While prior research has primarily focused on improving FC accuracy, little attention has been given to the robustness of these agents to perturbations in their input. We introduce a benchmark assessing FC robustness in two key areas: resilience to naturalistic query variations, and stability in function calling when the toolkit expands with semantically related tools. Evaluating best-performing FC models on a carefully expanded subset of the Berkeley function calling leaderboard (BFCL), we identify critical weaknesses in existing evaluation methodologies, and highlight areas for improvement in real-world agentic deployments.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 20 of its 27 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

20 of 27 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets26 Jun
  2. 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning16 Jan
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  4. 2023Gorilla: Large Language Model Connected with Massive APIs24 May

Other references · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  7. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  8. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 188
  9. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 111
  10. 2023Conversational Health Agents: A Personalized LLM-Powered Agent Frameworkcited by 92
  11. 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 71
  12. 2024Large Language Model Agent in Financial Trading: A Surveycited by 60
4 more
  1. 2022Measuring Reliability of Large Language Models through Semantic Consistencycited by 38
  2. 2023Predicting Question-Answering Performance of Large Language Models through Semantic Consistencycited by 28
  3. 2024A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarioscited by 8
  4. 2024Content Knowledge Identification with Multi-Agent Large Language Models (LLMs)cited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 13.

19 papers.

Sort:

By area:

  1. 2025OpenApps: Simulating Environment Variations to Measure UI-Agent Reliabilitycited by 12
  2. 2025Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agentscited by 10
  3. 2026Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Modelscited by 10
  4. 2025Dynamic Tool Dependency Retrieval for Lightweight Function Callingcited by 7
  5. 2026How Many Tools Should an LLM Agent See? A Chance-Corrected Answercited by 7
  6. 2026When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agentscited by 6
  7. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system securitycited by 5
  8. 2026The "I Don't Know" Filter: Enhancing Agentic Reliability in Function Callingcited by 2
  9. 2025A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistantscited by 1
  10. 2025Assertion-Conditioned Compliance: A Provenance-Aware Vulnerability in Multi-Turn Tool-Calling Agentscited by 1
  11. 2026MAVEN: Improving Generalization in Agentic Tool Callingcited by 1
  12. 2026The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspectivecited by 1
7 more
  1. 2026SAP: State-Guided Data Synthesis with Argument Provenance for Multi-Turn Tool Usecited by 1
  2. 2025On Generalization in Agentic Tool Calling: CoreThink Agentic Reasoner and MAVEN Datasetcited by 0
  3. 2026From Interpretability to Control: Insights from Six Years of the TrustNLP Workshopcited by 0
  4. 2026Beyond Fluent Generation: A CPU Reliability Benchmark for MCP-Style Tool Calling in Sub-2B Small Language Models for Edge Deploymentcited by 0
  5. 2026MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agentscited by 0
  6. 2026Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventionscited by 0
  7. 2026One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustnesscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Hammer: Robust Function-Calling for On-Device Language Models via Function Masking
  2. 2026Small Reasoning Models are Instruction Followers in Function Calling
  3. 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
  4. 2026How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
  5. 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
  6. 2026From Task Solving to Robust Real-World Adaptation in LLM Agents
  7. 2026AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling
  8. 2026Beyond Perfect APIs: A Comprehensive Evaluation of LLM Agents Under Real-World API Complexity