# IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts

arXiv: https://arxiv.org/abs/2606.09908 · HTML: https://mlsanity.com/paper/2606.09908/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 53 references that are in the map.
- First posted on arXiv 6 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action](https://mlsanity.com/paper/2409.00138/) (2024)
- [The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies](https://mlsanity.com/paper/2407.19354/) (2024)
- [Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents](https://mlsanity.com/paper/2411.09523/) (2024)
- [Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild](https://mlsanity.com/paper/2407.11438/) (2024)
- [LLM-PBE: Assessing Data Privacy in Large Language Models](https://mlsanity.com/paper/2408.12787/) (2024)
- [CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Data](https://mlsanity.com/paper/2409.13903/) (2024)
- [PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance](https://mlsanity.com/paper/2502.17041/) (2025)
- [CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs](https://mlsanity.com/paper/2511.14937/) (2025)
- [Extracting books from production language models](https://mlsanity.com/paper/2601.02671/) (2026)
- [Understanding Privacy Norms Around LLM-Based Chatbots: A Contextual Integrity Perspective](https://mlsanity.com/paper/2508.06760/) (2025)
- [Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metric](https://mlsanity.com/paper/2409.03735/) (2024)

## Built on by (most cited first, 1 of 1)

- [PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agents](https://mlsanity.com/paper/2609.35937/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are becoming widely deployed as personal AI assistants with access to sensitive user data, making privacy a major challenge for their design and evaluation. Prior work focuses mainly on individual-level risks, overlooking interdependent privacy (IDP)--where one person's data may be revealed by others without their knowledge or consent. We address this gap by introducing IDP-Bench: the first LLM benchmark for IDP scenarios, grounded in the Contextual Integrity (CI) framework. We evaluate eight open-source LLMs on their understanding of IDP scenarios across three levels of IDP reasoning using two LLM judges. Results show strong co-ownership recognition (6/8 models exceed 90%) but persistent weaknesses in identifying CI parameters (information attribute, primary subject) and IDP-specific parameters such as secondary subjects, where 7/8 models score below 74%. Models also struggle to judge sharing appropriateness (5/8 scoring below 77%). While the ability to judge the appropriateness of sharing improves with scale, performance tends to decline in smaller models, and prompt sensitivity remains high on IDP-specific questions--highlighting the need for more targeted study of IDP in LLM privacy research. Data & code available hrefhttps://github.com/tisl-lab/InterdependentPrivacyBenchhere.