# Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

arXiv: https://arxiv.org/abs/2604.06409 · HTML: https://mlsanity.com/paper/2604.06409/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 7 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action](https://mlsanity.com/paper/2409.00138/) (2024)
- [On Protecting the Data Privacy of Large Language Models (LLMs): A Survey](https://mlsanity.com/paper/2403.05156/) (2024)
- [AirGapAgent: Protecting Privacy-Conscious Conversational Agents](https://mlsanity.com/paper/2405.05175/) (2024)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents](https://mlsanity.com/paper/2509.17488/) (2025)
- [Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Models](https://mlsanity.com/paper/2409.13474/) (2024)
- [SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation](https://mlsanity.com/paper/2506.12699/) (2025)
- [Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymization](https://mlsanity.com/paper/2306.05561/) (2023)
- [How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing](https://mlsanity.com/paper/2205.10095/) (2022)
- [Operationalizing Data Minimization for Privacy-Preserving LLM Prompting](https://mlsanity.com/paper/2510.03662/) (2025)
- [Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents](https://mlsanity.com/paper/2506.10171/) (2025)
- [Toward sensitive document release with privacy guarantees](https://mlsanity.com/paper/1701.00436/) (2017)
- [Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Services](https://mlsanity.com/paper/2510.27016/) (2025)
- [User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios](https://mlsanity.com/paper/2510.20721/) (2025)

## Abstract (arXiv)

LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private. Existing systems support only suppression (omitting sensitive information) and generalization (replacing information with an abstraction), and are typically evaluated on single isolated messages, leaving both the strategy space and evaluation setting incomplete. We formalize privacy-preserving LLM communication as an Information Sufficiency (IS) task, introduce free-text pseudonymization as a third strategy that replaces sensitive attributes with functionally equivalent alternatives, and propose a conversational evaluation protocol that assesses strategies under realistic multi-turn follow-up pressure. Across 792 scenarios spanning three power-relation types (institutional, peer, intimate) and three sensitivity categories (discrimination risk, social cost, boundary), we evaluate seven frontier LLMs on privacy at two granularities, covertness, and utility. Pseudonymization yields the strongest privacytextendash utility tradeoff overall, and single-message evaluation systematically underestimates leakage, with generalization losing up to 16.3 percentage points of privacy under follow-up.