# Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models

arXiv: https://arxiv.org/abs/2601.05366 · HTML: https://mlsanity.com/paper/2601.05366/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 17 of 17)

- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://mlsanity.com/paper/2309.10691/) (2023)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation](https://mlsanity.com/paper/2104.07412/) (2021)
- [T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step](https://mlsanity.com/paper/2312.14033/) (2023)
- [VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use](https://mlsanity.com/paper/2505.19255/) (2025)
- [AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2507.21836/) (2025)
- [A Survey on Data Augmentation in Large Model Era](https://mlsanity.com/paper/2401.15422/) (2024)
- [Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model](https://mlsanity.com/paper/2502.08820/) (2025)
- [On the Robustness of Agentic Function Calling](https://mlsanity.com/paper/2504.00914/) (2025)
- [X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents](https://mlsanity.com/paper/2306.17674/) (2023)

## Built on by (most cited first, 10 of 10)

- [Repair, Not Improvement: Decomposing Constrained Decoding in Tool-Call Abstention](https://mlsanity.com/paper/2608.13959/) (2026)
- [OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents](https://mlsanity.com/paper/2608.08775/) (2026)
- [The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective](https://mlsanity.com/paper/2606.07017/) (2026)
- [An Actionable Diagnosis of Multilingual, Multi-Agent Planning Failures](https://mlsanity.com/paper/2608.03735/) (2026)
- [AndroidReality: How Far Are Mobile Agents from the Real World?](https://mlsanity.com/paper/2608.07775/) (2026)
- [From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix](https://mlsanity.com/paper/2609.01572/) (2026)
- [BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents](https://mlsanity.com/paper/2609.23490/) (2026)
- [Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR)](https://mlsanity.com/paper/2605.14126/) (2026)
- [Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models](https://mlsanity.com/paper/2606.15714/) (2026)
- [When Does Correction Become Repair? Mechanistic Auditing of Internal Interventions in Tool-Using LLMs](https://mlsanity.com/paper/2609.36138/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed as agents that invoke external tools through structured function calls. While recent work reports strong tool-calling performance under standard English-centric evaluations, the robustness of tool calling under multilingual user interactions remains underexplored. In this work, we introduce MLCL, a diagnostic benchmark, and conduct a systematic evaluation of multilingual tool calling across Chinese, Hindi, and the low-resource language Igbo. Through fine-grained error analysis, we show that many failures occur despite correct intent understanding and tool selection. We identify parameter value language mismatch as a dominant failure mode, where models generate semantically appropriate parameter values in the user's language, violating language-invariant execution conventions. We further evaluate several inference-time system strategies and find that while these strategies substantially reduce language-induced execution errors, none of them can fully recover English-level performance.