Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inference
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) such as ChatGPT can infer personal attributes from seemingly innocuous text, raising privacy risks beyond memorized data leakage. While prior work has demonstrated these risks, little is known about how users estimate and respond. We conducted a survey with 240 U.S. participants who judged text snippets for inference risks, reported concern levels, and attempted rewrites to block inference. We compared their rewrites with those generated by ChatGPT and Rescriber, a state-of-the-art sanitization tool. Results show that participants struggled to anticipate inference, performing a little better than chance. User rewrites were effective in just 28% of cases - better than Rescriber but worse than ChatGPT. We examined our participants' rewriting strategies, and observed that while paraphrasing was the most common strategy it is also the least effective; instead abstraction and adding ambiguity were more successful. Our work highlights the importance of inference-aware design in LLM interactions.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 35 of its 68 references that are in the map. First posted on arXiv 15 Sep 2025.
Builds on
35 of 68 references.
References · 35
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2021Membership Inference Attacks From First Principlescited by 839
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
- 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
- 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
- 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
- 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
- 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
- 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
23 more
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
- 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
- 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 89
- 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
- 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
- 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 43
- 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 38
- 2024Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Modelscited by 36
- 2024Private Attribute Inference from Images with Vision-Language Modelscited by 35
- 2024A Synthetic Dataset for Personal Attribute Inferencecited by 35
- 2024CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applicationscited by 33
- 2024PII-Compass: Guiding LLM training data extraction prompts towards the target PII via groundingcited by 22
- 2022Canary Extraction in Natural Language Understanding Modelscited by 20
- 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
- 2024Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Interventioncited by 15
- 2025Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Healthcited by 12
- 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 10
- 2024Listening to the Voices: Describing Ethical Caveats of Conversational User Interfaces According to Experts and Frequent Userscited by 8
- 2024Understanding Public Perceptions of AI Conversational Agents: A Cross-Cultural Analysiscited by 7
- 2025Understanding Users' Privacy Perceptions Towards LLM's RAG-based Memorycited by 6
- 2023Tell me, what are you most afraid of? Exploring the Effects of Agent Representation on Information Disclosure in Human-Chatbot Interactioncited by 5
- 2021Privacy Concerns in Chatbot Interactions: When to Trust and When to Worrycited by 4
- 2025Through Their Eyes: User Perceptions on Sensitive Attribute Inference of Social Media Videos by Visual Language Modelscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.