mlsanityData to 7 Oct 2026

Operationalizing Contextual Integrity in Privacy-Conscious Assistants

arXiv 2408.02373 ↗First posted 5 Aug 2024cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Advanced AI assistants combine frontier LLMs and tool access to autonomously perform complex tasks on behalf of users. While the helpfulness of such assistants can increase dramatically with access to user information including emails and documents, this raises privacy concerns about assistants sharing inappropriate information with third parties without user supervision. To steer information-sharing assistants to behave in accordance with privacy expectations, we propose to operationalize contextual integrity (CI), a framework that equates privacy with the appropriate flow of information in a given context. In particular, we design and evaluate a number of strategies to steer assistants' information-sharing actions to be CI compliant. Our evaluation is based on a novel form filling benchmark composed of human annotations of common webform applications, and it reveals that prompting frontier LLMs to perform CI-based reasoning yields strong results.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 5 Aug 2024.

Builds on

27 of 49 references.

References · 27

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023Mistral 7Bcited by 3,065
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  9. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  10. 2022PAL: Program-aided Language Modelscited by 635
  11. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  12. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
15 more
  1. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  2. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  3. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  4. 2021Internet-Augmented Dialogue Generationcited by 217
  5. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  6. 2022TALM: Tool Augmented Language Modelscited by 197
  7. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  8. 2021Generating Datasets with Pretrained Language Modelscited by 192
  9. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  10. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  11. 2024The Ethics of Advanced AI Assistantscited by 103
  12. 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agentscited by 90
  13. 2023Moral Foundations of Large Language Modelscited by 84
  14. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  15. 2024Measuring Social Norms of Large Language Modelscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 18.

36 papers.

Sort:

By area:

  1. 2025Defeating Prompt Injections by Designcited by 256
  2. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 82
  3. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 79
  4. 2024CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Datacited by 34
  5. 2025Firewalls to Secure Dynamic LLM Agentic Networkscited by 34
  6. 2025PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliancecited by 34
  7. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  8. 2024Permissive Information-Flow Analysis for Large Language Modelscited by 22
  9. 2025CASE-Bench: Context-Aware SafEty Benchmark for Large Language Modelscited by 17
  10. 20251-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoningcited by 16
  11. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 15
  12. 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agentcited by 14
24 more
  1. 2025ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversationscited by 14
  2. 2025Privacy Reasoning in Ambiguous Contextscited by 11
  3. 2025Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learningcited by 7
  4. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system securitycited by 5
  5. 2025Can Large Language Models Really Recognize Your Name?cited by 4
  6. 2025Personalizing Agent Privacy Decisions via Logical Entailmentcited by 4
  7. 2026Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agentscited by 4
  8. 2025Not My Agent, Not My Boundary? Elicitation of Personal Privacy Boundaries in AI-Delegated Information Sharingcited by 3
  9. 2026Intent-Governed Tool Authorization for AI Agentscited by 3
  10. 2026Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agentscited by 3
  11. 2024AI Delegates with a Dual Focus: Ensuring Privacy and Strategic Self-Disclosurecited by 2
  12. 2026Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representationscited by 2
  13. 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agentscited by 2
  14. 2026MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversationscited by 2
  15. 2025SoK: Data Minimization in Machine Learningcited by 1
  16. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agentscited by 1
  17. 2026PrivScope: Task-scoped Disclosure Control for Hybrid Agentic Systemscited by 1
  18. 2026Efficient and Sound Probabilistic Verification for AI Agentscited by 1
  19. 2026ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliancecited by 0
  20. 2026It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMscited by 0
  21. 2026ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failurescited by 0
  22. 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
  23. 2026REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editingcited by 0
  24. 2026Share No More Than the Request Requires: Federated Disclosure for Perspective-Aware AIcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems
  2. 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
  3. 2026IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts
  4. 2026CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents
  5. 2026From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management
  6. 2026Reinforcing privacy reasoning in LLMs via normative simulacra from fiction
  7. 2026AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows
  8. 2025Position: Human-Robot Interaction in Embodied Intelligence Demands a Shift From Static Privacy Controls to Dynamic Learning ↗