# CAPED: Context-Aware Privacy Exposure Defense for Mobile GUI Agents

arXiv: https://arxiv.org/abs/2606.12666 · HTML: https://mlsanity.com/paper/2606.12666/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 30 of 32)

- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception](https://mlsanity.com/paper/2401.16158/) (2024)
- [OmniParser for Pure Vision Based GUI Agent](https://mlsanity.com/paper/2408.00203/) (2024)
- [On the Effects of Data Scale on UI Control Agents](https://mlsanity.com/paper/2406.03679/) (2024)
- [Mobile-Agent-v3: Fundamental Agents for GUI Automation](https://mlsanity.com/paper/2508.15144/) (2025)
- [Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration](https://mlsanity.com/paper/2406.01014/) (2024)
- [EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage](https://mlsanity.com/paper/2409.11295/) (2024)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks](https://mlsanity.com/paper/2501.11733/) (2025)
- [PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System](https://mlsanity.com/paper/2206.03001/) (2022)
- [Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images](https://mlsanity.com/paper/1703.10660/) (2017)
- [Unveiling Privacy Risks in LLM Agent Memory](https://mlsanity.com/paper/2502.13172/) (2025)
- [AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents](https://mlsanity.com/paper/2503.09780/) (2025)
- [Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions](https://mlsanity.com/paper/2408.02544/) (2024)
- [Private Attribute Inference from Images with Vision-Language Models](https://mlsanity.com/paper/2404.10618/) (2024)
- [Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution](https://mlsanity.com/paper/2506.01055/) (2025)
- [Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images](https://mlsanity.com/paper/1712.01066/) (2017)
- [MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments](https://mlsanity.com/paper/2506.01616/) (2025)
- [A3: Android Agent Arena for Mobile GUI Agents with Essential-State Procedural Evaluation](https://mlsanity.com/paper/2501.01149/) (2025)
- [AppVLM: A Lightweight Vision Language Model for Online App Control](https://mlsanity.com/paper/2502.06395/) (2025)
- [Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Models](https://mlsanity.com/paper/2504.19373/) (2025)
- [Visual Content Privacy Protection: A Survey](https://mlsanity.com/paper/2303.16552/) (2023)
- [Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents](https://mlsanity.com/paper/2508.19493/) (2025)
- [Anonymization-Enhanced Privacy Protection for Mobile GUI Agents: Available but Invisible](https://mlsanity.com/paper/2602.10139/) (2026)
- [CAPID: Context-Aware PII Detection for Question-Answering Systems](https://mlsanity.com/paper/2602.10074/) (2026)
- [WebPII: Benchmarking Visual PII Detection for Computer-Use Agents](https://mlsanity.com/paper/2603.17357/) (2026)
- [PRISM-XR: Empowering Privacy-Aware XR Collaboration with Multimodal Large Language Models](https://mlsanity.com/paper/2602.10154/) (2026)

## Built on by (most cited first, 1 of 1)

- [PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agents](https://mlsanity.com/paper/2610.16778/) (2026)

## Abstract (arXiv)

Screenshot-based mobile GUI agents can operate ordinary smartphone apps through the same visual interface as a human user, but this capability also turns every screen observation into a privacy boundary. During normal task execution, screenshots may expose contacts, messages, photos, files, recommendations, health cues, and other sensitive context that is unrelated to the user's request. We call this problem incidental visual privacy exposure. It is difficult to address with existing defenses: text anonymization misses many visual and inferential cues, while generic privacy masking can remove the evidence and controls that a GUI agent needs to complete the task. This paper presents CAPED, a context-aware pre-upload exposure control layer for mobile GUI agents. CAPED is designed as a phone-side protection layer: before screenshots are released to a remote multimodal agent, it extracts task requirements, uses screen context as a privacy prior, parses visible UI elements, and selectively exposes only content needed for the current task while masking incidental private content. We evaluate CAPED on AndroidWorld for broad task utility and with a controlled 28-task seeded privacy evaluation used as a measurement instrument for trajectory-level incidental leakage. In this seeded evaluation, Full CAPED reduces success-conditioned weighted seeded leakage from 0.766 under raw screenshots to 0.268 while preserving high task utility. A broader AndroidWorld run shows a remaining prototype-level utility cost, but the results show that task-driven selective exposure can reduce incidental visual leakage before screenshots are released to a remote GUI agent.