# Human-Centred LLM Privacy Audits: Findings and Frictions

arXiv: https://arxiv.org/abs/2603.12094 · HTML: https://mlsanity.com/paper/2603.12094/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 63 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Are Large Pre-Trained Language Models Leaking Your Personal Information?](https://mlsanity.com/paper/2205.12628/) (2022)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [Targeting the Benchmark: On Methodology in Current Natural Language Processing Research](https://mlsanity.com/paper/2007.04792/) (2020)
- [Understanding accountability in algorithmic supply chains](https://mlsanity.com/paper/2304.14749/) (2023)
- [Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice](https://mlsanity.com/paper/2210.03709/) (2022)
- [Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP](https://mlsanity.com/paper/2405.17159/) (2024)
- [Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially](https://mlsanity.com/paper/2302.04720/) (2023)
- [WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI](https://mlsanity.com/paper/2501.01397/) (2025)
- [Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations](https://mlsanity.com/paper/2503.01623/) (2025)
- [Generative AI and Perceptual Harms: Who's Suspected of using LLMs?](https://mlsanity.com/paper/2410.00906/) (2024)
- [Minerva: A Programmable Memory Test Benchmark for Language Models](https://mlsanity.com/paper/2502.03358/) (2025)
- ["I've talked to ChatGPT about my issues last night.": Examining Mental Health Conversations with Large Language Models through Reddit Analysis](https://mlsanity.com/paper/2504.20320/) (2025)
- [When LLMs Struggle: Reference-less Translation Evaluation for Low-resource Languages](https://mlsanity.com/paper/2501.04473/) (2025)
- ["Create a Fear of Missing Out" -- ChatGPT Implements Unsolicited Deceptive Designs in Generated Websites Without Warning](https://mlsanity.com/paper/2411.03108/) (2024)
- [Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data](https://mlsanity.com/paper/2503.01630/) (2025)
- [Social Bias in Popular Question-Answering Benchmarks](https://mlsanity.com/paper/2505.15553/) (2025)
- [What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests](https://mlsanity.com/paper/2507.11128/) (2025)
- [Breaking the Benchmark: Revealing LLM Bias via Minimal Contextual Augmentation](https://mlsanity.com/paper/2510.23921/) (2025)

## Built on by (most cited first, 1 of 1)

- [No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation](https://mlsanity.com/paper/2608.21206/) (2026)

## Abstract (arXiv)

Large language models (LLMs) learn statistical associations from massive training corpora and user interactions, and deployed systems can surface or infer information about individuals. Yet people lack practical ways to inspect what a model associates with their name. We report interim findings from an ongoing study and introduce LMP2, a browser-based self-audit tool. In two user studies (Ntotal=458), GPT-4o predicts 11 of 50 features for everyday people with ≥60% accuracy, and participants report wanting control over LLM-generated associations despite not considering all outputs privacy violations. To validate our probing method, we evaluate eight LLMs on public figures and non-existent names, observing clear separation between stable name-conditioned associations and model defaults. Our findings also contribute to exposing a broader generative AI evaluation crisis: when outputs are probabilistic, context-dependent, and user-mediated through elicitation, what model--individual associations even include is under-specified and operationalisation relies on crafting probes and metrics that are hard to validate or compare. To move towards reliable, actionable human-centred LLM privacy audits, we identify nine frictions that emerged in our study and offer recommendations for future work and the design of human-centred LLM privacy audits.