# Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks

arXiv: https://arxiv.org/abs/2603.04364 · HTML: https://mlsanity.com/paper/2603.04364/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 18 of 18)

- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://mlsanity.com/paper/2401.01335/) (2024)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration](https://mlsanity.com/paper/1802.08802/) (2018)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [Self-Play Preference Optimization for Language Model Alignment](https://mlsanity.com/paper/2405.00675/) (2024)
- [EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage](https://mlsanity.com/paper/2409.11295/) (2024)
- [Self-playing Adversarial Language Game Enhances LLM Reasoning](https://mlsanity.com/paper/2404.10642/) (2024)
- [AdvAgent: Controllable Blackbox Red-teaming on Web Agents](https://mlsanity.com/paper/2410.17401/) (2024)
- [RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments](https://mlsanity.com/paper/2505.21936/) (2025)
- [Lessons from Defending Gemini Against Indirect Prompt Injections](https://mlsanity.com/paper/2505.14534/) (2025)
- [Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks](https://mlsanity.com/paper/2402.00626/) (2024)
- [Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography](https://mlsanity.com/paper/2405.14169/) (2024)
- [Adversarial Reinforcement Learning for Large Language Model Agent Safety](https://mlsanity.com/paper/2510.05442/) (2025)
- [Typographic Attacks in a Multi-Image Setting](https://mlsanity.com/paper/2502.08193/) (2025)
- [AI Safety in Practice: Enhancing Adversarial Robustness in Multimodal Image Captioning](https://mlsanity.com/paper/2407.21174/) (2024)

## Built on by (most cited first, 2 of 2)

- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model](https://mlsanity.com/paper/2610.08773/) (2026)

## Abstract (arXiv)

Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet their dual-stream architecture opens an underexplored attack surface: an adversary who injects content into the webpage DOM simultaneously corrupts both observation channels with a consistent deceptive narrative. Our vulnerability analysis on MiniWob++ reveals that attacks including a visual component far outperform text-only injections, exposing critical gaps in text-centric VLM safety training. Motivated by this finding, we propose Dual-Modality Multi-Stage Adversarial Safety Training (DMAST), a framework that formalizes the agent-attacker interaction as a two-player general-sum Markov game and co-trains both players through a three-stage pipeline: (1) imitation learning from a strong teacher model, (2) oracle-guided supervised fine-tuning that uses a novel zero-acknowledgment strategy to instill task-focused reasoning under adversarial noise, and (3) adversarial reinforcement learning via Group Relative Policy Optimization (GRPO) self-play. On out-of-distribution tasks, DMAST nearly halves the attack success rate (41.2%rightarrow21.4%) while raising task completion by over 60% relative (6.2%rightarrow10.2%). Our approach outperforms established training-based defenses and complements prompt-based defenses, demonstrating genuine co-evolutionary progress and robust generalization to complex, unseen environments. Code is available at https://github.com/huajianduzhuo-code/DMASTofficial.