mlsanityData to 14 Oct 2026

Eta Given Delta: Defining LLM Tool Efficiency With Marginal Tool Utility

arXiv 2607.14108 ↗First posted 7 May 2026cs.CL cs.AI cs.SEcited by 0

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

This paper introduces tool efficiency, a new quantitative metric to evaluate the rate of useful tool calls in an LLM agent trajectory. To ensure that tool efficiency is well-defined, we also introduce marginal tool utility, a new quantitative metric defined per tool call indicating whether a tool is useful or whether it can be safely removed from the tool suite without affecting accuracy while increasing tool efficiency; in this paper, we determine the sign of marginal tool utility for each tool call in a trajectory using LLM-as-a-Judge. While much prior work has been done to develop techniques that improve tool use by LLMs and design evaluation methods measuring efficiency indirectly using accuracy as a proxy, our work is centered on measuring efficiency directly via the quantitative metric proposed in this paper in post hoc trajectory analyses. It is our intention that this work contributes to the frontier of LLM evaluation research as a springboard for future benchmark designs and agent harness engineering (specifically with regards to creating lean tool suites) that optimize for metrics that complement but are distinct from accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 50 of its 60 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

50 of 60 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers31 Jan
  2. 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering22 Oct
  3. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Models15 Sep
  4. 2024TinyAgent: Function Calling at the Edge1 Sep
  5. 2024WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark for Large Language Models2 Jul
  6. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models12 Mar
  7. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  9. 2023Large Language Models as Tool Makers26 May
  10. 2023ART: Automatic multi-step reasoning and tool-use for large language models16 Mar
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  11. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  12. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
27 more
  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  3. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  4. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  5. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
  6. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
  7. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
  8. 2022Multi-lingual Evaluation of Code Generation Modelscited by 159
  9. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  10. 2025Recursive Language Modelscited by 104
  11. 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Executioncited by 85
  12. 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 76
  13. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  14. 2024Dated Data: Tracing Knowledge Cutoffs in Large Language Modelscited by 51
  15. 2024Offline Training of Language Model Agents with Functions as Learnable Weightscited by 48
  16. 2025LLM Agents Making Agent Toolscited by 48
  17. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 34
  18. 2025Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Modelscited by 27
  19. 2026Learning to Continually Learn via Meta-learning Agentic Memory Designscited by 18
  20. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMscited by 17
  21. 2026Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoningcited by 15
  22. 2024ChemToolAgent: The Impact of Tools on Language Agents for Chemistry Problem Solvingcited by 11
  23. 2025SAND-Math: Using LLMs to Generate Novel, Difficult and Useful Mathematics Questions and Answerscited by 11
  24. 2016Spectrum-based Software Fault Localization: A Survey of Techniques, Advances, and Challengescited by 9
  25. 2026On Calibration of Large Language Models: From Response To Capabilitycited by 7
  26. 2025DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoningcited by 5
  27. 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applicationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Toward Architecture-Aware Evaluation Metrics for LLM Agents
  2. 2026Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses
  3. 2026Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents
  4. 2026ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents
  5. 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
  6. 2026ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox
  7. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents
  8. 2026MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agents