# What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents

arXiv: https://arxiv.org/abs/2609.39989 · HTML: https://mlsanity.com/paper/2609.39989/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 294 of its 357 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 30 of 294)

- [Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners](https://mlsanity.com/paper/2307.01928/) (2023)
- [Multi-agent Architecture Search via Agentic Supernet](https://mlsanity.com/paper/2502.04180/) (2025)
- [ADaPT: As-Needed Decomposition and Planning with Language Models](https://mlsanity.com/paper/2311.05772/) (2023)
- [Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities](https://mlsanity.com/paper/2307.13565/) (2023)
- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Explore until Confident: Efficient Exploration for Embodied Question Answering](https://mlsanity.com/paper/2403.15941/) (2024)
- [SkillRouter: Skill Routing for LLM Agents at Scale](https://mlsanity.com/paper/2603.22455/) (2026)
- [TinyAgent: Function Calling at the Edge](https://mlsanity.com/paper/2409.00608/) (2024)
- [Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval](https://mlsanity.com/paper/2408.01875/) (2024)
- [SciToolAgent: A Knowledge Graph-Driven Scientific Agent for Multi-Tool Integration](https://mlsanity.com/paper/2507.20280/) (2025)
- [Web-Shepherd: Advancing PRMs for Reinforcing Web Agents](https://mlsanity.com/paper/2505.15277/) (2025)
- [FIRE: Fact-checking with Iterative Retrieval and Verification](https://mlsanity.com/paper/2411.00784/) (2024)
- [ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agents](https://mlsanity.com/paper/2505.06416/) (2025)
- [MCP-Zero: Active Tool Discovery for Autonomous LLM Agents](https://mlsanity.com/paper/2506.01056/) (2025)
- [Towards Uncertainty-Aware Language Agent](https://mlsanity.com/paper/2401.14016/) (2024)
- [When Agents go Astray: Course-Correcting SWE Agents with PRMs](https://mlsanity.com/paper/2509.02360/) (2025)
- [BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism](https://mlsanity.com/paper/2505.20660/) (2025)
- [ExpSeek: Self-Triggered Experience Seeking for Web Agents](https://mlsanity.com/paper/2601.08605/) (2026)
- [SkillOrchestra: Learning to Route Agents via Skill Transfer](https://mlsanity.com/paper/2602.19672/) (2026)
- [VeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verification](https://mlsanity.com/paper/2503.18492/) (2025)
- [SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents](https://mlsanity.com/paper/2512.07850/) (2025)
- [Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systems](https://mlsanity.com/paper/2502.00409/) (2025)
- [DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal](https://mlsanity.com/paper/2503.14269/) (2025)
- [BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks](https://mlsanity.com/paper/2411.07464/) (2024)
- [ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering](https://mlsanity.com/paper/2510.20036/) (2025)
- [Agentic Uncertainty Quantification](https://mlsanity.com/paper/2601.15703/) (2026)
- [Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents](https://mlsanity.com/paper/2603.26233/) (2026)
- [LLM Agents Already Know When to Call Tools -- Even Without Reasoning](https://mlsanity.com/paper/2605.09252/) (2026)
- [E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing](https://mlsanity.com/paper/2512.03109/) (2025)
- [Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents](https://mlsanity.com/paper/2602.16699/) (2026)

## Abstract (arXiv)

Background. A component replacement in a language-model agent changes an execution trajectory, potentially altering later observations, resource use, and recovery opportunities. Different evidence is needed to assess its task-level benefit and the contribution of local decision quality. Methods. This critical scoping review maps 348 studies and examines 90 comparison records: 88 from 40 included studies and two from supplementary studies. Eight purposively selected cases structure the synthesis around the replaced decision, executed conditions, measurement comparability, controls, and remaining explanations. Results. Of 222 studies reporting local decision metrics, 142 also report measured task endpoints and 49 report proxies. These counts identify studies that report both types of measurement, without establishing that the measurements come from matched comparisons. Outcome Monitors reports a package-level completion gain whose attribution to detector quality remains limited; First-chunk selection reports a local improvement assessed against an offline proxy endpoint; Evidence-Carrying Termination reports fewer premature unsupported terminations and completion non-inferiority, without establishing completion superiority. Cross-case analysis identifies three candidate mechanisms involving recovery and disruption, intervention timing, and downstream use. Attribution and deployment depend on the comparison controls, label definitions, and information available to the controller. Conclusions. The review distinguishes the task-level benefit of a component replacement from the contribution of local decision quality and derives eight claim-specific reporting items. Neither online execution nor simultaneous gains in local and task metrics alone establish that better local decisions explain the task-level gain.