It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing
Not on the map. Why?
Abstract (arXiv)
Aligning AI systems with human privacy preferences requires understanding individuals' nuanced disclosure behaviors beyond general privacy norms. Yet eliciting these boundaries remains challenging because they involve context-dependent trade-offs. We present an elicitation approach that probes individuals' privacy boundaries through a discriminative task. Through a between-subjects study with varying communication roles and delegation conditions, we collected 1,650 privacy boundaries from 169 participants across 60 scenarios. We examined how communication roles, AI delegation, and individual differences influence boundaries. Results show that communication roles significantly influence sensitivity to disclosure granularity or identifiability, depending on whose private information is shared; AI delegation and individuals' need for privacy heighten sensitivity to disclosed identifiers. Our analysis further reveals how AI delegation shifts privacy preferences across contexts, highlighting new complexities in AI-mediated communication. We advocate eliciting nuanced privacy boundaries within real-world data flows by considering roles and AI delegation, and enabling user involvement in guiding privacy-aware AI behavior.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 139 references that are in the map. First posted on arXiv 26 Sep 2025.
Builds on
25 of 139 references.
References · 25
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
- 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
- 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
- 2024A Roadmap to Pluralistic Alignmentcited by 184
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2023Eliciting Human Preferences with Language Modelscited by 92
- 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 69
- 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
- 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
- 2024Operationalizing Contextual Integrity in Privacy-Conscious Assistantscited by 37
- 2024Aligning Language Models with Demonstrated Feedbackcited by 34
- 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 31
13 more
- 2024Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theorycited by 18
- 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
- 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agentcited by 14
- 2025Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Healthcited by 13
- 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Datacited by 10
- 2025Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platformscited by 9
- 2025Scrolling in the Deep: Analysing Contextual Influences on Intervention Effectiveness during Infinite Scrolling on Social Mediacited by 5
- 2022Stop the Spread: A Contextual Integrity Perspective on the Appropriateness of COVID-19 Vaccination Certificatescited by 4
- 2025Artificial Intelligence in Deliberation: The AI Penalty and the Emergence of a New Deliberative Dividecited by 4
- 2025Towards Aligning Personalized Conversational Recommendation Agents with Users' Privacy Preferencescited by 4
- 2014A Utility-Theoretic Approach to Privacy in Online Servicescited by 3
- 2024User Privacy Harms and Risks in Conversational AI: A Proposed Frameworkcited by 3
- 2024Personalizing Privacy Protection With Individuals' Regulatory Focus: Would You Preserve or Enhance Your Information Privacy?cited by 1
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.