mlsanityData to 7 Oct 2026

Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions

arXiv 2408.02544 ↗First posted 5 Aug 2024cs.CLcited by 55

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

This paper investigates the faithfulness of multimodal large language model (MLLM) agents in a graphical user interface (GUI) environment, aiming to address the research question of whether multimodal GUI agents can be distracted by environmental context. A general scenario is proposed where both the user and the agent are benign, and the environment, while not malicious, contains unrelated content. A wide range of MLLMs are evaluated as GUI agents using a simulated dataset, following three working patterns with different levels of perception. Experimental results reveal that even the most powerful models, whether generalist agents or specialist GUI agents, are susceptible to distractions. While recent studies predominantly focus on the helpfulness of agents, our findings first indicate that these agents are prone to environmental distractions. Furthermore, we implement an adversarial environment injection and analyze the approach to improve faithfulness, calling for a collective focus on this important topic.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 62 references that are in the map. First posted on arXiv 5 Aug 2024.

Builds on

40 of 62 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
  2. 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs8 Apr
  3. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
  4. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation13 Nov

Other references · 36

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  6. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  7. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  8. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  9. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
24 more
  1. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  2. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  3. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  4. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  5. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  7. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  8. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  9. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  10. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  11. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  12. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  13. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  14. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  15. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  16. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
  17. 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 147
  18. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  19. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  20. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  21. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  22. 2024TrustAgent: Towards Safe and Trustworthy LLM-based Agentscited by 62
  23. 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injectioncited by 53
  24. 2024SoFA: Shielded On-the-fly Alignment via Priority Rule Followingcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 36; 2026: 14.

55 papers.

Sort:

By area:

  1. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  2. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 156
  3. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attackscited by 140
  4. 2024Attacking Vision-Language Computer Agents via Pop-upscited by 112
  5. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agentscited by 112
  6. 2024GUI Agents: A Surveycited by 101
  7. 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
  8. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  9. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  10. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  11. 2025RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agentscited by 34
  12. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
38 more
  1. 2024AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generationcited by 31
  2. 2025macOSWorld: A Multilingual Interactive Benchmark for GUI Agentscited by 31
  3. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 28
  4. 2025OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agentscited by 25
  5. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  6. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environmentscited by 23
  7. 2025Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attackscited by 21
  8. 2025Digi-Q: Learning Q-Value Functions for Training Device-Control Agentscited by 20
  9. 2024Dynamic Planning for LLM-based Graphical User Interface Automationcited by 19
  10. 2026Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agentscited by 18
  11. 2025MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Themcited by 16
  12. 2025Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agentscited by 12
  13. 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?cited by 12
  14. 2025Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Scheduling Systemcited by 11
  15. 2025Generalizability of Large Language Model-Based Agents: A Comprehensive Surveycited by 10
  16. 2025OpenApps: Simulating Environment Variations to Measure UI-Agent Reliabilitycited by 10
  17. 2025AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Deliverycited by 9
  18. 2025ColorBench: Benchmarking Mobile Agents with Graph-Structured Framework for Complex Long-Horizon Taskscited by 9
  19. 2025In-Context Defense in Computer Agents: An Empirical Studycited by 8
  20. 2025SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agentscited by 8
  21. 2025Are Large Language Models Sensitive to the Motives Behind Communication?cited by 7
  22. 2025LaSM: Layer-wise Scaling Mechanism for Defending Pop-up Attack on GUI Agentscited by 6
  23. 2025Mobile GUI Agents under Real-world Threats: Are We There Yet?cited by 5
  24. 2025Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environmentscited by 5
  25. 2025Textual-to-Visual Iterative Self-Verification for Slide Generationcited by 4
  26. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
  27. 2025Superplatforms Have to Attack AI Agentscited by 3
  28. 2025Agent-ScanKit: Unraveling Memory and Reasoning of Multimodal Agents via Sensitivity Perturbationscited by 3
  29. 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 3
  30. 2026Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanizationcited by 3
  31. 2026WAAA! Web Adversaries Against Agentic Browserscited by 3
  32. 2026AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptionscited by 3
  33. 2025How Deep is Love in LLMs' Hearts? Exploring Semantic Size in Human-like Cognitioncited by 2
  34. 2026Under the Influence: Quantifying Persuasion and Vigilance in Large Language Modelscited by 2
  35. 2026AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agentscited by 2
  36. 2026Faithful Mobile GUI Agents with Guided Advantage Estimatorcited by 1
  37. 2026When Routine Chats Turn Toxic: Unintended Long-Term State Poisoning in Personalized Agentscited by 1
  38. 2026HLL: Can Agents Cross Humanity's Last Line of Verification?cited by 1

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024GUI Agents with Foundation Models: A Comprehensive Survey
  2. 2025API Agents vs. GUI Agents: Divergence and Convergence
  3. 2025A Survey on (M)LLM-Based GUI Agents
  4. 2026GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents
  5. 2024GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
  6. 2025The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections
  7. 2025The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs
  8. 2026VISUALSKILL: Multimodal Skills for Computer-Use Agents