When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents
Not on the map. Why?
Abstract (arXiv)
Tool-augmented data agents rely on tool outputs for analytical decisions. Yet successful execution can return plausible but incorrect evidence, requiring agents to decide whether to trust or verify it. Understanding this failure requires examining both the evidence obtained through checking and the answer ultimately adopted. We introduce ToxicBench to measure checking and adoption under numerical, label, schema, and retrieval errors, pairing clean and poisoned observations over fixed source data. In the 118-task GPT evaluation across three adapters, poisoning lowers task success by 26 to 39 percentage points. Ordinary retries help under one-shot poisoning, whereas repeated poisoning reveals wrong-answer adoption after checking. Controls on three public tables isolate how supplied evidence affects recovery. After freezing the scorer, we compare its judgments with human annotations on 200 trajectories, finding 96% task-success agreement. Human judgments support retry gains over Base and confirm adoption after checking on audited tasks. We release trajectories, versioned scoring, and reference and delivery audits. These findings highlight evidence availability and answer selection as complementary dimensions of agent reliability.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
26 of 34 references.
References · 26
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2025SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Modelcited by 31
- 2024Metacognitive Retrieval-Augmented Large Language Modelscited by 26
14 more
- 2024Tools Fail: Detecting Silent Errors in Faulty Toolscited by 23
- 2025DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecyclecited by 22
- 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Casescited by 21
- 2025Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agentscited by 20
- 2025MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Toolscited by 15
- 2026The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agentscited by 14
- 2026MCP-ITP: An Automated Framework for Implicit Tool Poisoning in MCPcited by 13
- 2026Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agentscited by 9
- 2026AgentAtlas: Beyond Outcome Leaderboards for LLM Agentscited by 9
- 2025RADAR: Benchmarking Language Models on Imperfect Tabular Datacited by 5
- 2025ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systemscited by 5
- 2026Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliabilitycited by 5
- 2026Sell Me This Stock: Unsafe Recommendation Drift in LLM Agentscited by 2
- 2026ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agentscited by 2
Built on by
Not cited yet.