mlsanityData to 14 Oct 2026

PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance

arXiv 2502.17041 ↗First posted 24 Feb 2025cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Recent advancements in generative large language models (LLMs) have enabled wider applicability, accessibility, and flexibility. However, their reliability and trustworthiness are still in doubt, especially for concerns regarding individuals' data privacy. Great efforts have been made on privacy by building various evaluation benchmarks to study LLMs' privacy awareness and robustness from their generated outputs to their hidden representations. Unfortunately, most of these works adopt a narrow formulation of privacy and only investigate personally identifiable information (PII). In this paper, we follow the merit of the Contextual Integrity (CI) theory, which posits that privacy evaluation should not only cover the transmitted attributes but also encompass the whole relevant social context through private information flows. We present PrivaCI-Bench, a comprehensive contextual privacy evaluation benchmark targeted at legal compliance to cover well-annotated privacy and safety regulations, real court cases, privacy policies, and synthetic data built from the official toolkit to study LLMs' privacy and safety compliance. We evaluate the latest LLMs, including the recent reasoner models QwQ-32B and Deepseek R1. Our experimental results suggest that though LLMs can effectively capture key CI parameters inside a given context, they still require further advancements for privacy compliance.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 20 of its 40 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

20 of 40 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  11. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  12. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
8 more
  1. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  2. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
  3. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 45
  4. 2024GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theorycited by 37
  5. 2024Operationalizing Contextual Integrity in Privacy-Conscious Assistantscited by 37
  6. 2024CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Datacited by 37
  7. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  8. 2024Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theorycited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 23.

37 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  2. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 34
  3. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  4. 2025Searching for Privacy Risks in LLM Agents via Simulationcited by 22
  5. 2025Position: Privacy Is Not Just Memorization!cited by 17
  6. 2025ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversationscited by 16
  7. 2025Privacy Reasoning in Ambiguous Contextscited by 11
  8. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  9. 2026Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failurescited by 8
  10. 2025AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliancecited by 6
  11. 2025SafeMT: Multi-turn Safety for Multimodal Language Modelscited by 6
  12. 2026Automated Researchers Can Mitigate Well-characterized Alignment Failurescited by 5
25 more
  1. 2025Personalizing Agent Privacy Decisions via Logical Entailmentcited by 4
  2. 2026Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?cited by 3
  3. 2025Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliancecited by 2
  4. 2026Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representationscited by 2
  5. 2026PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systemscited by 2
  6. 2025User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarioscited by 1
  7. 2026Computational Compliance for AI Regulation: Blueprint for a New Research Domaincited by 1
  8. 2026Safer Policy Compliance with Dynamic Epistemic Fallbackcited by 1
  9. 2026MedPriv-Bench: Benchmarking the Privacy-Utility Trade-off of Large Language Models in Medical Open-Ended Question Answeringcited by 1
  10. 2026WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Taskscited by 1
  11. 2026It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMscited by 1
  12. 2026IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contextscited by 1
  13. 2026ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposurecited by 1
  14. 2025HKGAI-V1: Towards Regional Sovereign Large Language Model for Hong Kongcited by 0
  15. 2026Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignmentcited by 0
  16. 2026Which private attributes do VLMs agree on and predict well?cited by 0
  17. 2026Policy Compliance of User Requests in Natural Language for AI Systemscited by 0
  18. 2026ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliancecited by 0
  19. 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
  20. 2026DAIS: Dependency-Aware Intermediate QA Supervision for Complex Reasoningcited by 0
  21. 2026CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignmentcited by 0
  22. 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safetycited by 0
  23. 2026PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversationscited by 0
  24. 2026Jev-Guard: Fast Routing and Auditing Agent Actions over Editable Policy Treescited by 0
  25. 2026AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning
  2. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  3. 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data
  4. 2025A Survey on Privacy Risks and Protection in Large Language Models
  5. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
  6. 2026PIA-Bench: Towards Automated Privacy Impact Assessment with Large Language Models ↗
  7. 2025PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty
  8. 2025Auditing M-LLMs for Privacy Risks: A Synthetic Benchmark and Evaluation Framework ↗