DualTAP: A Dual-Task Adversarial Protector for Mobile MLLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The reliance of mobile GUI agents on Multimodal Large Language Models (MLLMs) introduces a severe privacy vulnerability: screenshots containing Personally Identifiable Information (PII) are often sent to untrusted, third-party routers. These routers can exploit their own MLLMs to mine this data, violating user privacy. Existing privacy perturbations fail the critical dual challenge of this scenario: protecting PII from the router's MLLM while simultaneously preserving task utility for the agent's MLLM. To address this gap, we propose the Dual-Task Adversarial Protector (DualTAP), a novel framework that, for the first time, explicitly decouples these conflicting objectives. DualTAP trains a lightweight generator using two key innovations: (i) a contrastive attention module that precisely identifies and targets only the PII-sensitive regions, and (ii) a dual-task adversarial objective that simultaneously minimizes a task-preservation loss (to maintain agent utility) and a privacy-interference loss (to suppress PII leakage). To facilitate this study, we introduce PrivScreen, a new dataset of annotated mobile screenshots designed specifically for this dual-task evaluation. Comprehensive experiments on six diverse MLLMs (e.g., GPT-5) demonstrate DualTAP's state-of-the-art protection. It reduces the average privacy leakage rate by 31.6 percentage points (a 3.0x relative improvement) while, critically, maintaining an 80.8% task success rate - a negligible drop from the 83.6% unprotected baseline. DualTAP presents the first viable solution to the privacy-utility trade-off in mobile MLLM agents.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 26 of its 36 references that are in the map. First posted on arXiv 17 Nov 2025.
Builds on
26 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments2 Jun
- 2025WebInject: Prompt Injection Attack to Web Agents16 May
Other references · 24
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
- 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 284
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
- 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
- 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
- 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
- 2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 144
- 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
12 more
- 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 111
- 2024GUI Agents: A Surveycited by 103
- 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 88
- 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agentscited by 63
- 2025A Survey on (M)LLM-Based GUI Agentscited by 62
- 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 43
- 2025MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agentscited by 38
- 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
- 2025MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Toolscited by 32
- 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
- 2025LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applicationscited by 14
- 2025VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Modelscited by 4
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
- 2026Mind the Gap: Action Rebinding Attacks against Android GUI Agentscited by 6
- 2026Anonymization-Enhanced Privacy Protection for Mobile GUI Agents: Available but Invisiblecited by 6
- 2026Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecyclecited by 2
- 2026Defending Wearable VLMs Against Private Attribute Inferencecited by 1
- 2026SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chainscited by 0
- 2026PriMobiBench: Characterizing Visual Privacy Leakage in VLM-Driven Mobile GUI Agentscited by 0
- 2026SAGE: Efficient Protected Vision-Language Inference on Smart Glassescited by 0
- 2026PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agentscited by 0
- 2026PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agents12 Oct 2026
- 2026SAGE: Efficient Protected Vision-Language Inference on Smart Glasses8 Oct 2026
- 2026PriMobiBench: Characterizing Visual Privacy Leakage in VLM-Driven Mobile GUI Agents12 Sep 2026
- 2026Defending Wearable VLMs Against Private Attribute Inference26 Aug 2026
- 2026SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chains7 Aug 2026
- 2026Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle5 Aug 2026
- 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution29 Apr 2026
- 2026Anonymization-Enhanced Privacy Protection for Mobile GUI Agents: Available but Invisible8 Feb 2026
- 2026Mind the Gap: Action Rebinding Attacks against Android GUI Agents18 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
- 2026Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Models
- 2026MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content
- 2026POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents
- 2026Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges
- 2023Can Language Models be Instructed to Protect Personal Information?
- 2026Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
- 2026When Visual Privacy Protection Meets Multimodal Large Language Models