SlowBA: An efficiency backdoor attack towards VLM-based GUI agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low latency. While existing research on GUI-agent security mainly focuses on manipulating action correctness, the security risks related to response efficiency remain largely unexplored. In this paper, we introduce SlowBA, a novel backdoor attack that targets the responsiveness of VLM-based GUI agents. The key idea is to manipulate response latency by inducing excessively long reasoning chains under specific trigger patterns. To achieve this, we propose a two-stage reward-level backdoor injection (RBI) strategy that first aligns the long-response format and then learns trigger-aware activation through reinforcement learning. In addition, we design realistic pop-up windows as triggers that naturally appear in GUI environments, improving the stealthiness of the attack. Extensive experiments across multiple datasets and baselines demonstrate that SlowBA can significantly increase response length and latency while largely preserving task accuracy. The attack remains effective even with a small poisoning ratio and under several defense settings. These findings reveal a previously overlooked security vulnerability in GUI agents and highlight the need for defenses that consider both action correctness and response efficiency. Code can be found in https://github.com/tu-tuing/SlowBA.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 61 of its 74 references that are in the map. First posted on arXiv 9 Mar 2026.
Builds on
61 of 74 references.
Closest ancestors · 2
Same direction, newest first
- 2025VisualTrap: A Stealthy Backdoor Attack on GUI Agents via Visual Grounding Manipulation9 Jul
- 2025Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents20 May
Other references · 59
- 2023GPT-4 Technical Reportcited by 18,132
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Visual Instruction Tuningcited by 7,792
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
38 more
- 2018Spectral Signatures in Backdoor Attackscited by 457
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
- 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
- 2024On the Effects of Data Scale on UI Control Agentscited by 187
- 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 183
- 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Webcited by 156
- 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 112
- 2018Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examplescited by 98
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
- 2018Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compressioncited by 74
- 2024CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automationcited by 70
- 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Imagescited by 69
- 2024ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Areacited by 68
- 2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 62
- 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
- 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
- 2024MC-LLaVA: Multi-Concept Personalized Vision-Language Modelcited by 53
- 2025MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environmentcited by 51
- 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
- 2022NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Modelscited by 47
- 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
- 2025ZeroGUI: Automating Online GUI Learning at Zero Human Costcited by 44
- 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
- 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
- 2024TrojVLM: Backdoor Attack Against Vision Language Modelscited by 35
- 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Reviewcited by 34
- 2024Backdooring Vision-Language Models with Out-Of-Distribution Datacited by 31
- 2024Physical Backdoor Attack can Jeopardize Driving with Vision-Large-Language Modelscited by 30
- 2023Intelligent Virtual Assistants with LLM-based Process Automationcited by 26
- 2025PolySkill: Learning Generalizable Skills Through Polymorphic Abstractioncited by 25
- 2025Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigationcited by 22
- 2026Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representationcited by 21
- 2025BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Modelscited by 15
- 2025VeriOS: Query-Driven Proactive Human-Agent-GUI Interaction for Trustworthy OS Agentscited by 15
- 2025Backdoor Attack on Vision Language Models with Stealthy Semantic Manipulationcited by 13
- 2025IAG: Input-aware Backdoor Attack on VLM-based Visual Groundingcited by 11
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026PEARL: Personalized Streaming Video Understanding Modelcited by 8
- 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agentscited by 0
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability8 May 2026
- 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agents27 Apr 2026
- 2026PEARL: Personalized Streaming Video Understanding Model20 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Inject Once Survive Later: Backdooring Vision-Language-Action Models to Persist Through Downstream Fine-tuning
- 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustness
- 2025DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models
- 2025Goal-oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects
- 2026Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systems
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
- 2025LaSM: Layer-wise Scaling Mechanism for Defending Pop-up Attack on GUI Agents
- 2026AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents