MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Deep research agents increasingly combine private local documents with external tools like web retrieval, creating a privacy risk: an agent's external queries may leak sensitive information from its local context. This risk is amplified by the mosaic effect, where individual queries may appear harmless but become revealing in aggregate. We introduce MosaicLeaks, a benchmark of 1,001 multi-hop deep research tasks that chain private enterprise documents and a public web corpus, forcing agents to make external queries that depend on local information. We evaluate leakage with an adversary LLM that observes only the agent's external queries and attempts to infer private information at three levels: the agent's research intent, answers to specific private questions and verifiable claims about the enterprise documents. We find that models across families and sizes frequently leak at all three levels, that zero-shot privacy prompting reduces but does not eliminate leakage and that reinforcement learning for task performance alone worsens leakage. To address this, we propose Privacy-Aware Deep Research (PA-DR), an RL framework that combines situational rewards for task success with a learned privacy classifier to provide dense credit assignment over both per-query and mosaic-level leakage. Training Qwen3-4B-Instruct with PA-DR improves accuracy from 48.7% to 58.7% and reduces answer and full-information leakage from 34.0% to 9.9%.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 21 of its 26 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
21 of 26 references.
Closest ancestors · 3
Same direction, newest first
- 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation18 Dec
- 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration16 Sep
- 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
Other references · 18
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
- 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
- 2025DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Researchcited by 114
- 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agentscited by 92
- 2025DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Researchcited by 53
- 2025Deep Research Bench: Evaluating AI Web Research Agentscited by 32
- 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
6 more
- 2025DRBench: A Realistic Benchmark for Enterprise Deep Researchcited by 22
- 2025Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analyticscited by 20
- 2026DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agentcited by 10
- 2025HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searchescited by 7
- 2026InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seekingcited by 5
- 2026SPILLage: Agentic Oversharing on the Webcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning
- 2026Differential Privacy in Generative AI Agents: Analysis and Optimal Tradeoffs
- 2026OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents
- 2025PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AI
- 2026ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Querying
- 2026Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity
- 2025MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation