# In-Context Defense in Computer Agents: An Empirical Study

arXiv: https://arxiv.org/abs/2503.09241 · HTML: https://mlsanity.com/paper/2503.09241/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked](https://mlsanity.com/paper/2308.07308/) (2023)
- [GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation](https://mlsanity.com/paper/2311.07562/) (2023)
- [Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents](https://mlsanity.com/paper/2402.11208/) (2024)
- [Dissecting Adversarial Robustness of Multimodal LM Agents](https://mlsanity.com/paper/2406.12814/) (2024)
- [Attacking Vision-Language Computer Agents via Pop-ups](https://mlsanity.com/paper/2411.02391/) (2024)
- [Efficient Adversarial Training in LLMs with Continuous Attacks](https://mlsanity.com/paper/2405.15589/) (2024)
- [The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use](https://mlsanity.com/paper/2411.10323/) (2024)
- [AdvAgent: Controllable Blackbox Red-teaming on Web Agents](https://mlsanity.com/paper/2410.17401/) (2024)
- [Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions](https://mlsanity.com/paper/2408.02544/) (2024)
- [MMInA: Benchmarking Multihop Multimodal Internet Agents](https://mlsanity.com/paper/2404.09992/) (2024)
- [AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations](https://mlsanity.com/paper/2411.13451/) (2024)

## Built on by (most cited first, 10 of 10)

- [OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://mlsanity.com/paper/2506.14866/) (2025)
- [macOSWorld: A Multilingual Interactive Benchmark for GUI Agents](https://mlsanity.com/paper/2506.04135/) (2025)
- [CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents](https://mlsanity.com/paper/2601.09923/) (2026)
- [LaSM: Layer-wise Scaling Mechanism for Defending Pop-up Attack on GUI Agents](https://mlsanity.com/paper/2507.10610/) (2025)
- [Mind the Gap: Action Rebinding Attacks against Android GUI Agents](https://mlsanity.com/paper/2601.12349/) (2026)
- [Prismata: Confining Cross-Site Prompt Injection in Web Agents](https://mlsanity.com/paper/2607.08147/) (2026)
- [Mind the Gap: Evaluating Model- and Agentic-Level Vulnerabilities in LLMs with Action Graphs](https://mlsanity.com/paper/2509.04802/) (2025)
- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [Securing Computer-Use Agents Against Branch Steering Attacks](https://mlsanity.com/paper/2610.03089/) (2026)
- [SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents](https://mlsanity.com/paper/2610.18206/) (2026)

## Abstract (arXiv)

Computer agents powered by vision-language models (VLMs) have significantly advanced human-computer interaction, enabling users to perform complex tasks through natural language instructions. However, these agents are vulnerable to context deception attacks, an emerging threat where adversaries embed misleading content into the agent's operational environment, such as a pop-up window containing deceptive instructions. Existing defenses, such as instructing agents to ignore deceptive elements, have proven largely ineffective. As the first systematic study on protecting computer agents, we introduce textbf{in-context defense}, leveraging in-context learning and chain-of-thought (CoT) reasoning to counter such attacks. Our approach involves augmenting the agent's context with a small set of carefully curated exemplars containing both malicious environments and corresponding defensive responses. These exemplars guide the agent to first perform explicit defensive reasoning before action planning, reducing susceptibility to deceptive attacks. Experiments demonstrate the effectiveness of our method, reducing attack success rates by 91.2% on pop-up window attacks, 74.6% on average on environment injection attacks, while achieving 100% successful defenses against distracting advertisements. Our findings highlight that (1) defensive reasoning must precede action planning for optimal performance, and (2) a minimal number of exemplars (fewer than three) is sufficient to induce an agent's defensive behavior.