# Human-Centered Privacy Research in the Age of Large Language Models

arXiv: https://arxiv.org/abs/2402.01994 · HTML: https://mlsanity.com/paper/2402.01994/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 3 Feb 2024.

## Builds on (most cited first, 8 of 8)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [Counterfactual Memorization in Neural Language Models](https://mlsanity.com/paper/2112.12938/) (2021)
- [Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?](https://mlsanity.com/paper/2204.04741/) (2022)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [Can Language Models be Instructed to Protect Personal Information?](https://mlsanity.com/paper/2310.02224/) (2023)

## Built on by (most cited first, 9 of 9)

- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbots](https://mlsanity.com/paper/2410.11876/) (2024)
- [Embedding Large Language Models into Extended Reality: Opportunities and Challenges for Inclusion, Engagement, and Privacy](https://mlsanity.com/paper/2402.03907/) (2024)
- [LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data](https://mlsanity.com/paper/2508.12158/) (2025)
- [DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Tasks](https://mlsanity.com/paper/2502.00270/) (2025)
- [Beyond Data Privacy: New Privacy Risks for Large Language Models](https://mlsanity.com/paper/2509.14278/) (2025)
- [StorySage: Conversational Autobiography Writing Powered by a Multi-Agent Framework](https://mlsanity.com/paper/2506.14159/) (2025)
- [Designing Beyond Language: Sociotechnical Barriers in AI Health Technologies for Limited English Proficiency](https://mlsanity.com/paper/2511.07277/) (2025)
- [One Size Fits All? A Modular Adaptive Sanitization Kit (MASK) for Customizable Privacy-Preserving Phone Scam Detection](https://mlsanity.com/paper/2510.18493/) (2025)

## Abstract (arXiv)

The emergence of large language models (LLMs), and their increased use in user-facing systems, has led to substantial privacy concerns. To date, research on these privacy concerns has been model-centered: exploring how LLMs lead to privacy risks like memorization, or can be used to infer personal characteristics about people from their content. We argue that there is a need for more research focusing on the human aspect of these privacy issues: e.g., research on how design paradigms for LLMs affect users' disclosure behaviors, users' mental models and preferences for privacy controls, and the design of tools, systems, and artifacts that empower end-users to reclaim ownership over their personal data. To build usable, efficient, and privacy-friendly systems powered by these models with imperfect privacy properties, our goal is to initiate discussions to outline an agenda for conducting human-centered research on privacy issues in LLM-powered systems. This Special Interest Group (SIG) aims to bring together researchers with backgrounds in usable security and privacy, human-AI collaboration, NLP, or any other related domains to share their perspectives and experiences on this problem, to help our community establish a collective understanding of the challenges, research opportunities, research methods, and strategies to collaborate with researchers outside of HCI.