mlsanityData to 7 Oct 2026

When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents

arXiv 2605.08828 ↗First posted 9 May 2026cs.AIcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large language model agents increasingly operate through environment-facing scaffolds that expose files, web pages, APIs, and logs. These observations influence tool use, state tracking, and action sequencing, yet their reliability and authority are often uncertain. Environmental grounding is therefore a systems-level problem involving context admission, evidence provenance, freshness checking, verification policy, action gating, and model reasoning. Existing agent benchmarks mainly evaluate task capability or specific attacks such as prompt injection and memory poisoning, but they under-specify a fundamental reliability question: whether agents remain grounded in the true environment state when observations are stale, incorrect, or malicious. We introduce EnvTrustBench, an agentic framework for benchmarking this failure mode. We define an evidence-grounding defect (EGD) as a behavioral failure in which an agent treats an environment-facing claim as sufficient evidence for action without resolving it against available current evidence, leading to a task-incorrect false path under the true environment state. Given a task scenario, EnvTrustBench generates the workspace, environment, agent-facing objective, and validation oracle, executes the evaluated agent, records its action-observation trajectory and final state, and applies the oracle to produce a verdict. Using 6 LLM backbones and 5 widely used scaffolds, we evaluate 55 generated cases across 11 task scenarios, with each scenario expanded through five feedback-guided generation iterations. Results show that EGDs consistently emerge across operational workflows, highlighting environmental grounding as a core agent reliability problem with important security implications.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 17 of its 22 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

17 of 22 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem31 May
  2. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  3. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
  4. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  5. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  6. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  7. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 10

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  5. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  6. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  7. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  8. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  9. 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 62
  10. 2024Attacks on Third-Party APIs of Large Language Modelscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
  2. 2026Auditing Provenance Sensitivity in LLM Agent Action Selection
  3. 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
  4. 2026AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
  5. 2026AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
  6. 2026AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions
  7. 2026MAP-Graph: Provenance-Aware Shared Memory for Multi-Agent Workflows
  8. 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems