AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
The rapid adoption of mobile graphical user interface (GUI) agents, which autonomously control applications and operating systems (OS), exposes new system-level attack surfaces. Existing backdoors against web GUI agents and general GenAI models rely on environmental injection or deceptive pop-ups to mislead the agent operation. However, these techniques do not work on screenshots-based mobile GUI agents due to the challenges of restricted trigger design spaces, OS background interference, and conflicts in multiple trigger-action mappings. We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers (e.g., benign app icons in notifications). To address the underfitting caused by natural triggers and achieve accurate multi-target action redirection, we design a novel two-stage pipeline that first enhances the agent's sensitivity to subtle iconographic differences via contrastive learning, and then associates each trigger with a specific mobile GUI agent action through a backdoor post-training. Our extensive evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations. Furthermore, it is hard to visibly detect the benign-looking triggers and circumvents eight representative state-of-the-art defenses. These results expose an overlooked backdoor vector in mobile GUI agents, underscoring the need for defenses that scrutinize notification-conditioned behaviors and internal agent representations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 35 of its 46 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
35 of 46 references.
Closest ancestors · 11
Same direction, newest first
- 2025AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery27 May
- 2025Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents20 May
- 2025The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections15 Apr
- 2025Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attacks18 Feb
- 2025DemonAgent: Dynamically Encrypted Multi-Backdoor Implantation Attack on LLM-based Agent18 Feb
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
- 2024Attacking Vision-Language Computer Agents via Pop-ups4 Nov
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents5 Jun
- 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents17 Feb
Other references · 24
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
- 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2020Backdoor Learning: A Surveycited by 412
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
- 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
- 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
- 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
12 more
- 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
- 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
- 2024CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automationcited by 70
- 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 67
- 2024Understanding the Weakness of Large Language Model Agents within a Complex Android Environmentcited by 60
- 2024Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractionscited by 55
- 2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 53
- 2022Rethinking the Reverse-engineering of Trojan Triggerscited by 37
- 2022Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical Worldcited by 20
- 2024Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubscited by 16
- 2023BEAGLE: Forensics of Deep Learning Backdoor Attack for Better Defensecited by 15
- 2025On the Robustness of GUI Grounding Models Against Image Attackscited by 14
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025VisualTrap: A Stealthy Backdoor Attack on GUI Agents via Visual Grounding Manipulation
- 2026MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content
- 2026(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents
- 2026Temporal UI State Inconsistency in Desktop GUI Agents: Formalizing and Defending Against TOCTOU Attacks on Computer-Use Agents
- 2026Mind the Gap: Action Rebinding Attacks against Android GUI Agents
- 2026Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
- 2026MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps
- 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agents