Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
On the map: LLM agents and tool use → Tool learning and function calling
Abstract (arXiv)
Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply one. We isolate this post-failure decision with a benchmark of 1,024 items spanning 16 internal-system domains and eight tool-failure types, in which a tool call is enforced and the returned payload is guaranteed to be unusable. Under a deployment-style system prompt, 14.10% of responses are dishonest: the model either asserts a value the payload cannot support or declines while citing a fabricated policy or capability limit. The rate is governed almost entirely by whether the failure is signalled. When the tool returns status:error, dishonesty is absent (0.0%); when it returns status:ok with a redacted, corrupted, stale, malformed, empty or truncated value, dishonesty reaches 45.3%. The behaviour is not an artefact of our prompts: it appears under a neutral prompt (10.17%) and under the shipped prompt of every production agent framework we evaluate, reaching 24.67% under CrewAI's, and none of the nine frameworks we audit specifies what the model should do when a tool fails. Comparing prompt-level defences, we find that the operative variable is not deference to tool output but the absence of a named failure state. Appending a single sentence that requires the model to emit retrieval_status: OK or FAILED before answering reduces dishonesty from 14.10% to 0.87%, with one item of 688 worsening against 92 improving, and transfers unchanged into three foreign agent scaffolds. The emitted flag is faithful in 99.7-99.9% of declarations, giving a runtime detector that needs only a regular expression.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 35 of its 38 references that are in the map. First posted on arXiv 13 Sep 2026.
Builds on
35 of 38 references.
Closest ancestors · 5
Same direction, newest first
- 2026ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents6 Jul
- 2025When2Call: When (not) to Call Tools26 Apr
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
- 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb
Other references · 30
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
18 more
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
- 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
- 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
- 2023Do Large Language Models Know What They Don't Know?cited by 243
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
- 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
- 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 113
- 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 86
- 2023Context-faithful Prompting for Large Language Modelscited by 80
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 17
- 2026From Confident Closing to Silent Failure: Characterizing False Success in LLM Agentscited by 15
- 2026Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Whycited by 6
- 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirementscited by 1
- 2026Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Modelscited by 1
- 2026Guardrails as Scapegoats: Auditing Unfaithful Safety Refusals in Tool-Augmented LLM Agentscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Loud Failures, Quiet Failures: Fault Detection and Recovery in Tool-Using Language Model Agents
- 2026Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail
- 2026Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability
- 2026Agents Trust Tools Too Much: Measuring Reliance on Unreliable Tools
- 2026When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime
- 2026Tool Calling is Linearly Readable and Steerable in Language Models
- 2026Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents
- 2026Test-time reasoning effort and unauthorized tool use in language-model agents: a prespecified equivalence study