Judged Useless, Queried Anyway: Tool-Using Agents Rarely Turn Their Own Evidence Judgments into Stopping Decisions
Not on the map. Why?
Abstract (arXiv)
An agent whose tool keeps returning nothing useful should stop relying on it. In a retrieval environment with controlled source failures, we separate how agents judge results from what they do. We compare stopping at the same step after longer and shorter runs of results the agent judged useless; this contrast is zero for clock- or deadline-driven stopping. Where we record their judgments, the seven agents we test call a failing source's results useless 97-100% of the time, yet most of them rarely stop on that judgment. Prompt cues change when they stop but not what they stop on. Permission to answer from memory and a reasoning mode can bring early stops regardless of evidence, a stated budget moves the 7-8B models' stops to the deadline, and a stopping rule or call cost in the prompt is followed at most partly. Stopping follows the evidence only when the harness enforces an integration step that makes the agent answer after five consecutive results it judged useless. This step raises failing-source success for every model, keeps the stopping point fixed when the budget doubles, and needs no extra judgment call when the agent states its judgments. A pre-registered replication on 300 fresh questions confirms the dissociation and the rule's effect.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 5 Oct 2026.
Builds on
22 of 31 references.
References · 22
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2023Active Retrieval Augmented Generationcited by 516
- 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 41
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 17
- 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 15
- 2026Over-Searching in Search-Augmented Large Language Modelscited by 11
- 2025Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflowscited by 7
- 2026To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Interventioncited by 7
10 more
- 2026Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Usecited by 7
- 2026Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliabilitycited by 7
- 2026HALT: Verification-Aware Stopping for Retrieval-Augmented Search Agentscited by 3
- 2026When Deep Research Agents Stagnate: Enhancing Reasoning with Retrieval-Aware Agent Controlcited by 3
- 2026When May an Agent Stop? Evidence-Carrying Termination for Tool-Using LLMscited by 3
- 2026From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agentscited by 2
- 2026Diagnosing Search Behavior and Failure Modes in Long-Horizon Search Agentscited by 2
- 2026Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injectioncited by 2
- 2026From Inertia to Objectivity: Improving Deep Research Agents with Noise Isolationcited by 1
- 2026Agents Trust Tools Too Much: Measuring Reliance on Unreliable Toolscited by 1
Built on by
Not cited yet.