# Don't Click That: Teaching Web Agents to Resist Deceptive Interfaces

arXiv: https://arxiv.org/abs/2605.09497 · HTML: https://mlsanity.com/paper/2605.09497/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 10 May 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 17 of 17)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024)
- [Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents](https://mlsanity.com/paper/2408.07199/) (2024)
- [You Only Look at Screens: Multimodal Chain-of-Action Agents](https://mlsanity.com/paper/2309.11436/) (2023)
- [OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://mlsanity.com/paper/2506.14866/) (2025)
- [Fight Back Against Jailbreaking via Prompt Adversarial Tuning](https://mlsanity.com/paper/2402.06255/) (2024)
- [Factuality of Large Language Models: A Survey](https://mlsanity.com/paper/2402.02420/) (2024)
- [RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments](https://mlsanity.com/paper/2505.21936/) (2025)
- [Investigating the Impact of Dark Patterns on LLM-Based Web Agents](https://mlsanity.com/paper/2510.18113/) (2025)
- [Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications](https://mlsanity.com/paper/2308.05898/) (2023)
- [50 Shades of Deceptive Patterns: A Unified Taxonomy, Multimodal Detection, and Security Implications](https://mlsanity.com/paper/2501.13351/) (2025)
- [ProvenanceWidgets: A Library of UI Control Elements to Track and Dynamically Overlay Analytic Provenance](https://mlsanity.com/paper/2407.17431/) (2024)

## Built on by (most cited first, 1 of 1)

- [Before Acting, Change the State: Prospective State Intervention for Web Agents under Deceptive Interfaces](https://mlsanity.com/paper/2609.34974/) (2026)

## Abstract (arXiv)

Vision-language model (VLM) based web agents demonstrate impressive autonomous GUI interaction but remain vulnerable to deceptive interface elements. Existing approaches either detect deception without task integration or document attacks without proposing defenses. We formalize deception-aware web agent defense and propose DUDE (Deceptive UI Detector & Evaluator), a two-stage framework combining hybrid-reward learning with asymmetric penalties and experience summarization to distill failure patterns into transferable guidance. We introduce RUC (Real UI Clickboxes), a benchmark of 1,407 scenarios spanning four domains and deception categories. Experiments show DUDE reduces deception susceptibility by 53.8% while maintaining task performance, establishing an effective foundation for robust web agent deployment.