mlsanityData to 14 Oct 2026

Contextualized Privacy Defense for LLM Agents

arXiv 2603.02983 ↗First posted 3 Mar 2026cs.CR cs.AI cs.CLcited by 7

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

LLM agents increasingly act on users' personal information, yet existing privacy defenses remain limited in both design and adaptability. Most prior approaches rely on static or passive defenses, such as prompting and guarding. These paradigms are insufficient for supporting contextual, proactive privacy decisions in multi-step agent execution. We propose Contextualized Defense Instructing (CDI), a new privacy defense paradigm in which an instructor model generates step-specific, context-aware privacy guidance during execution, proactively shaping actions rather than merely constraining or vetoing them. Crucially, CDI is paired with an experience-driven optimization framework that trains the instructor via reinforcement learning (RL), where we convert failure trajectories with privacy violations into learning environments. We formalize baseline defenses and CDI as distinct intervention points in a canonical agent loop, and compare their privacy-helpfulness trade-offs within a unified simulation framework. Results show that our CDI consistently achieves a better balance between privacy preservation (94.2%) and helpfulness (80.6%) than baselines, with superior robustness to adversarial conditions and generalization.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 26 of its 37 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

26 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Searching for Privacy Risks in LLM Agents via Simulation14 Aug
  2. 20251-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning11 Aug
  3. 2025Progent: Securing AI Agents with Privilege Control16 Apr
  4. 2025Firewalls to Secure Dynamic LLM Agentic Networks3 Feb
  5. 2024When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs18 Oct
  6. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Extracting Training Data from Large Language Modelscited by 2,189
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  6. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  7. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  8. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  9. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  10. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  11. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  12. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
8 more
  1. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  2. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  3. 2024CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Followingcited by 33
  4. 2025CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMscited by 26
  5. 2025Eliciting Language Model Behaviors with Investigator Agentscited by 23
  6. 2024Defending Against Social Engineering Attacks in the Age of LLMscited by 15
  7. 2025PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databasescited by 6
  8. 2025Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Searchcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning
  2. 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
  3. 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training
  4. 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
  5. 2026It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs
  6. 2025Position Paper: Rethinking Privacy in RL for Sequential Decision-making in the Age of LLMs
  7. 2025Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents
  8. 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agents