mlsanityData to 14 Oct 2026

DualTAP: A Dual-Task Adversarial Protector for Mobile MLLM Agents

arXiv 2511.13248 ↗First posted 17 Nov 2025cs.CRcited by 9

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The reliance of mobile GUI agents on Multimodal Large Language Models (MLLMs) introduces a severe privacy vulnerability: screenshots containing Personally Identifiable Information (PII) are often sent to untrusted, third-party routers. These routers can exploit their own MLLMs to mine this data, violating user privacy. Existing privacy perturbations fail the critical dual challenge of this scenario: protecting PII from the router's MLLM while simultaneously preserving task utility for the agent's MLLM. To address this gap, we propose the Dual-Task Adversarial Protector (DualTAP), a novel framework that, for the first time, explicitly decouples these conflicting objectives. DualTAP trains a lightweight generator using two key innovations: (i) a contrastive attention module that precisely identifies and targets only the PII-sensitive regions, and (ii) a dual-task adversarial objective that simultaneously minimizes a task-preservation loss (to maintain agent utility) and a privacy-interference loss (to suppress PII leakage). To facilitate this study, we introduce PrivScreen, a new dataset of annotated mobile screenshots designed specifically for this dual-task evaluation. Comprehensive experiments on six diverse MLLMs (e.g., GPT-5) demonstrate DualTAP's state-of-the-art protection. It reduces the average privacy leakage rate by 31.6 percentage points (a 3.0x relative improvement) while, critically, maintaining an 80.8% task success rate - a negligible drop from the 83.6% unprotected baseline. DualTAP presents the first viable solution to the privacy-utility trade-off in mobile MLLM agents.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 26 of its 36 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

26 of 36 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments2 Jun
  2. 2025WebInject: Prompt Injection Attack to Web Agents16 May

Other references · 24

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  5. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  6. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 284
  7. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  8. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  9. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  10. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  11. 2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 144
  12. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
12 more
  1. 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 111
  2. 2024GUI Agents: A Surveycited by 103
  3. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 88
  4. 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agentscited by 63
  5. 2025A Survey on (M)LLM-Based GUI Agentscited by 62
  6. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 43
  7. 2025MCPWorld: A Unified Benchmarking Testbed for API, GUI, and Hybrid Computer Use Agentscited by 38
  8. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  9. 2025MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Toolscited by 32
  10. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  11. 2025LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applicationscited by 14
  12. 2025VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
  2. 2026Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Models
  3. 2026MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content
  4. 2026POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents
  5. 2026Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges
  6. 2023Can Language Models be Instructed to Protect Personal Information?
  7. 2026Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
  8. 2026When Visual Privacy Protection Meets Multimodal Large Language Models