mlsanityData to 7 Oct 2026

How Adversarial Environments Mislead Agentic AI?

arXiv 2604.18874 ↗First posted 20 Apr 2026cs.AIcited by 4

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Tool-integrated agents are deployed on the premise that external tools ground their outputs in reality. Yet this very reliance creates a critical attack surface. Current evaluations benchmark capability in benign settings, asking "can the agent use tools correctly" but never "what if the tools lie". We identify this Trust Gap: agents are evaluated for performance, not for skepticism. We formalize this vulnerability as Adversarial Environmental Injection (AEI), a threat model where adversaries compromise tool outputs to deceive agents. AEI constitutes environmental deception: constructing a "fake world" of poisoned search results and fabricated reference networks around unsuspecting agents. We operationalize this via POTEMKIN, a Model Context Protocol (MCP)-compatible harness for plug-and-play robustness testing. We identify two orthogonal attack surfaces: The Illusion (breadth attacks) poison retrieval to induce epistemic drift toward false beliefs, while The Maze (depth attacks) exploit structural traps to cause policy collapse into infinite loops. Across 11,000+ runs on five frontier agents, we find a stark robustness gap: resistance to one attack often increases vulnerability to the other, demonstrating that epistemic and navigational robustness are distinct capabilities.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 42 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

25 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agents20 Sep
  2. 2025Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools4 Aug
  3. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  11. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
10 more
  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  3. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  4. 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
  5. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  6. 2023Do Language Models Know When They're Hallucinating References?cited by 115
  7. 2023The Semantic Scholar Open Data Platformcited by 109
  8. 2025TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generationcited by 37
  9. 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Taskscited by 34
  10. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surfacecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures
  2. 2025EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks
  3. 2026Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection
  4. 2025Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attacks
  5. 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents
  6. 2026SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence
  7. 2026The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems
  8. 2026When the Agent Is the Adversary: Architectural Requirements for Agentic AI Containment After the April 2026 Frontier Model Escape