mlsanityData to 7 Oct 2026

Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents

arXiv 2509.17488 ↗First posted 22 Sep 2025cs.CR cs.AIcited by 33

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The increasing autonomy of LLM agents in handling sensitive communications, accelerated by Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, creates urgent privacy challenges. While recent work reveals significant gaps between LLMs' privacy Q&A performance and their agent behavior, existing benchmarks remain limited to static, simplified scenarios. We present PrivacyChecker, a model-agnostic, contextual integrity based mitigation approach that effectively reduces privacy leakage from 36.08% to 7.30% on DeepSeek-R1 and from 33.06% to 8.32% on GPT-4o, all while preserving task helpfulness. We also introduce PrivacyLens-Live, transforming static benchmarks into dynamic MCP and A2A environments that reveal substantially higher privacy risks in practical. Our modular mitigation approach integrates seamlessly into agent protocols through three deployment strategies, providing practical privacy protection for the emerging agentic ecosystem. Our data and code will be made available at https://aka.ms/privacy_in_action.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 15 of its 31 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

15 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb

Other references · 14

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  3. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  4. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  5. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
  6. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 108
  7. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  8. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
  9. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  10. 2023The Consensus Game: Language Model Generation via Equilibrium Searchcited by 35
  11. 2024CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Datacited by 34
  12. 2024Safely Learning with Private Data: A Federated Learning Framework for Large Language Modelcited by 11
2 more
  1. 2024Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Taskscited by 4
  2. 2024Privacy Evaluation Benchmarks for NLP Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 29.

33 papers.

Sort:

By area:

  1. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 26
  2. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 17
  3. 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigationcited by 11
  4. 2026CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agentscited by 10
  5. 2026AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflowscited by 9
  6. 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Trainingcited by 6
  7. 2026AgentRaft: Automated Detection of Data Over-Exposure in LLM Agentscited by 6
  8. 2026Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalizationcited by 5
  9. 2026POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agentscited by 4
  10. 2026MPCI-Bench: A Benchmark for Multimodal Pairwise Contextual Integrity Evaluation of Language Model Agentscited by 2
  11. 2026Minim: Privacy-Aware Minimal View for Agents via Trusted Local Sanitizationcited by 2
  12. 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agentscited by 2
21 more
  1. 2026MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversationscited by 2
  2. 2025User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarioscited by 1
  3. 2025AudAgent: Automated Auditing of Privacy Policy Compliance in AI Agentscited by 1
  4. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agentscited by 1
  5. 2026Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMscited by 1
  6. 2026MaskClaw: Edge-Side Personalized Privacy Arbitration for GUI Agents with Behavior-Driven Skill Evolutioncited by 1
  7. 2026ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposurecited by 1
  8. 2026Say Something Else: Rethinking Contextual Privacy as Information Sufficiencycited by 0
  9. 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
  10. 2026It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMscited by 0
  11. 2026OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agentscited by 0
  12. 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
  13. 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scalecited by 0
  14. 2026DP-MemView: A Memory Interface for Attribute-Level Transcript Privacy in Long-Term LLM Agentscited by 0
  15. 2026Rethinking Agent Security as a Networking Problemcited by 0
  16. 2026Inferring Hidden User Models from the Behavior of Personalized LLM Agentscited by 0
  17. 2026Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Controlcited by 0
  18. 2026ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessionscited by 0
  19. 2026PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversationscited by 0
  20. 2026CoSec: Benchmarking Agent Security in Communitiescited by 0
  21. 2026PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20251-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning
  2. 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems
  3. 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
  4. 2025MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation
  5. 2026AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy
  6. 2025Maris: A Formally Verifiable Privacy Policy Enforcement Paradigm for Multi-Agent Collaboration Systems
  7. 2025AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaboration
  8. 2025Searching for Privacy Risks in LLM Agents via Simulation