mlsanityData to 7 Oct 2026

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

arXiv 2607.27083 ↗First posted 29 Jul 2026cs.LG cs.AIcited by 2

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure. Routers and retrievers can rank candidate tools by relevance, but a ranking alone does not determine how many are worth selecting. Existing approaches leave acquisition under heterogeneous costs unaddressed. We formulate this decision as cost-aware marginal decision-focused stopping (CAM-DF) over ranked tool prefixes, with CAM-DF-lite as a compact interpretable variant. We train directly on the offline gap between stopping now and the best continuation: its sign labels the decision, its magnitude weights each error by the payoff at stake. We prove this objective is Bayes-aligned with the stopping target and that score-only rules are suboptimal under heterogeneous costs. We evaluate on 1,343 tasks across five tool-use domains. On τ-bench Retail, CAM-DF attains the highest payoff among deployable methods, with gains over a predict-then-threshold baseline across all five ranking sources and two cost regimes. Our approach is state-of-the-art under heterogeneous costs and high cost pressure, with larger gains under weaker rankings. In live execution, CAM-DF exposes the agent to 37% fewer tools than full access while maintaining comparable task success. The CAM-DF family is a lightweight pre-execution plugin that turns existing tool rankings into lower-cost acquisition decisions without fine-tuning the underlying LLM.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 26 of its 48 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

26 of 48 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026LLM Agents Already Know When to Call Tools -- Even Without Reasoning10 May
  2. 2026To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling1 May
  3. 2025When2Call: When (not) to Call Tools26 Apr
  4. 2025SMART: Self-Aware Agent for Tool Overuse Mitigation17 Feb
  5. 2024Budget-Constrained Tool Learning with Planning25 Feb
  6. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls6 Feb
  7. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  9. 2023Gorilla: Large Language Model Connected with Massive APIs24 May

Other references · 17

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  3. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  4. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  5. 2017Smart "Predict, then Optimize"cited by 332
  6. 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Settingcited by 47
  7. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 42
  8. 2025CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agentscited by 32
  9. 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agentscited by 16
  10. 2024CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planningcited by 14
  11. 2020Pandora's Box Problem with Order Constraintscited by 12
  12. 2020Choppy: Cut Transformer For Ranked List Truncationcited by 11
5 more
  1. 2024EcoAct: Economic Agent Determines When to Register What Actioncited by 11
  2. 2023Recent Developments in Pandora's Box Problem: Variants and Applicationscited by 5
  3. 2026Utility-Guided Agent Orchestration for Efficient LLM Tool Usecited by 4
  4. 2026TASR: Training-Free Adaptive Stopping for Iterative Retrievalcited by 4
  5. 2026Online Pandora's Box for Contextual LLM Cascadingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026What Does a Skill Actually Do? Estimands and Evaluation Validity for Tool and Skill Use in LLM Agents: A Critical Reviewcited by 0
  2. 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses
  2. 2026TRACER: Per-Tool Context Retention for LLM Agents via Consequence-Attributed Reinforcement Learning
  3. 2026Toollery: Scaling LLM Agents to Thousands of Skills and Tools
  4. 2026Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents
  5. 2026Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents
  6. 2026ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation
  7. 2026SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents
  8. 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents