Action-Space Shaping for LLM Agents: Measuring and Mitigating Tool-Schema Bias
On the map: LLM agents and tool use → Tool learning and function calling
Abstract (arXiv)
Large Language Models (LLMs) have shown strong performance on tool-use agentic tasks when given a fixed tool schema. Yet a tool schema is not the action space of an agent; it is merely one interface representation of it. The same executable action can be exposed through many different, functionally equivalent tool definitions, and an agent that has truly learned a task should behave consistently across them. We show that current agents often do not, a phenomenon we term schema bias. To study this systematically, we introduce an executable transformation framework that rewrites a native tool schema using nine operators, including merging and splitting tools, altering how a single tool is expressed, and distributing one action across several dependent calls. The tasks, executable actions, and reachable states remain fixed, so any change in success is attributable to the interface alone. Evaluating eleven LLMs, including two closed models, on up to 32 schema variants, we ask how large schema bias is, how it manifests, whether the difficulty of a schema variant can be predicted without a full evaluation, and whether training removes it. We find that schema bias is substantial even for the newest models: success rates range from complete failure to 97% depending solely on the schema. To reliably estimate schema difficulty, it requires running a small sample of the target queries. Training repairs a schema variant only when that variant appears in the training data.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 30 of its 39 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
30 of 39 references.
Closest ancestors · 8
Same direction, newest first
- 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL18 May
- 2025ACEBench: Who Wins the Match Point in Tool Usage?22 Jan
- 2024ToolACE: Winning the Points of LLM Function Calling2 Sep
- 2024Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks27 Jun
- 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets26 Jun
- 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning16 Jan
- 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
- 2023Gorilla: Large Language Model Connected with Massive APIs24 May
Other references · 22
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
10 more
- 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
- 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
- 2026Gemma 4 Technical Reportcited by 187
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
- 2020Action Space Shaping in Deep Reinforcement Learningcited by 45
- 2025Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Modelscited by 6
- 2026From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoningcited by 5
- 2024TOAD: Task-Oriented Automatic Dialogs with Diverse Response Stylescited by 4
- 2026Joint Optimization of Tool Creation and Use for Large Language Model Agentscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
- 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration
- 2026How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
- 2026DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agents
- 2026OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction
- 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents
- 2025AutoTool: Efficient Tool Selection for Large Language Model Agents
- 2025Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missions