mlsanityData to 14 Oct 2026

Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild

arXiv 2407.11438 ↗First posted 16 Jul 2024cs.CLcited by 47

Not on the map. Why?

Abstract (arXiv)

Measuring personal disclosures made in human-chatbot interactions can provide a better understanding of users' AI literacy and facilitate privacy research for large language models (LLMs). We run an extensive, fine-grained analysis on the personal disclosures made by real users to commercial GPT models, investigating the leakage of personally identifiable and sensitive information. To understand the contexts in which users disclose to chatbots, we develop a taxonomy of tasks and sensitive topics, based on qualitative and quantitative analysis of naturally occurring conversations. We discuss these potential privacy harms and observe that: (1) personally identifiable information (PII) appears in unexpected contexts such as in translation or code editing (48% and 16% of the time, respectively) and (2) PII detection alone is insufficient to capture the sensitive topics that are common in human-chatbot interactions, such as detailed sexual preferences or specific drug use habits. We believe that these high disclosure rates are of significant importance for researchers and data curators, and we call for the design of appropriate nudging mechanisms to help users moderate their interactions.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Builds on 22 of its 76 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

22 of 76 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  3. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  4. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  5. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  6. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 276
  7. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  8. 2023Efficient Large Language Models: A Surveycited by 186
  9. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
  10. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 129
  11. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  12. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
10 more
  1. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  2. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
  3. 2022UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Librarycited by 20
  4. 2023Position: Key Claims in LLM Research Have a Long Tail of Footnotescited by 20
  5. 2023Are Chatbots Ready for Privacy-Sensitive Applications? An Investigation into Input Regurgitation and Prompt-Induced Sanitizationcited by 17
  6. 2024Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Interventioncited by 15
  7. 2023NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMscited by 14
  8. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  9. 2022Are Neural Topic Models Broken?cited by 11
  10. 2023Identifying Self-Disclosures of Use, Misuse and Addiction in Community-based Social Media Postscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 21; 2026: 20.

47 papers.

Sort:

By area:

  1. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 72
  2. 2024Imprompter: Tricking LLM Agents into Improper Tool Usecited by 68
  3. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 36
  4. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  5. 2025Human Decision-making is Susceptible to AI-driven Manipulationcited by 19
  6. 2025Position: Privacy Is Not Just Memorization!cited by 17
  7. 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
  8. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 16
  9. 2025Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detectioncited by 13
  10. 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 13
  11. 2025Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Designcited by 11
  12. 2025User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policiescited by 9
35 more
  1. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  2. 2025Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMscited by 7
  3. 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacycited by 7
  4. 2026VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Modelscited by 7
  5. 2024Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metriccited by 6
  6. 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
  7. 2025Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learningcited by 6
  8. 2025Can Large Language Models Really Recognize Your Name?cited by 4
  9. 2025Show or Tell? Modeling the evolution of request-making in Human-LLM conversationscited by 3
  10. 2025The Subtle Art of Defection: Understanding Uncooperative Behaviors in LLM based Multi-Agent Systemscited by 3
  11. 2026PAAC: Privacy-Aware Agentic Device-Cloud Collaborationcited by 3
  12. 2024AI Delegates with a Dual Focus: Ensuring Privacy and Strategic Self-Disclosurecited by 2
  13. 2024Creation of AI-driven Smart Spaces for Enhanced Indoor Environments -- A Surveycited by 2
  14. 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inferencecited by 2
  15. 2025PATCH: Mitigating PII Leakage in Language Models with Privacy-Aware Targeted Circuit PatcHingcited by 2
  16. 2026Characterizing Narrative Content in Web-scale LLM Pretraining Datacited by 2
  17. 2026AI Fiction in the Wildcited by 2
  18. 2025User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarioscited by 1
  19. 2026Understanding Users' Privacy Reasoning and Behaviors During Chatbot Use to Support Meaningful Agency in Privacycited by 1
  20. 2026GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistantcited by 1
  21. 2026Adopt neq Adapt: Longitudinal Analyses of LLM Conversations in the Wildcited by 1
  22. 2026IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contextscited by 1
  23. 2026Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respondcited by 1
  24. 2025Investigating AI in Peer Support via Multi-Module System-Driven Embodied Conversational Agentscited by 0
  25. 2026Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agentscited by 0
  26. 2026Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Modelscited by 0
  27. 2026Reconstruction of Personally Identifiable Information from Proprietary Data in Supervised Fine-Tuned Modelscited by 0
  28. 2026Boundary-targeted Membership Inference Attacks on Safety Classifierscited by 0
  29. 2026PromptPET: Privacy-Utility Optimized Prompt Obfuscationcited by 0
  30. 2026The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generationcited by 0
  31. 2026Right Words, Wrong Moment: A Clinician-Grounded Analysis of Distress in 19,930 Conversations between Young People and ChatGPTcited by 0
  32. 2026PILLAR: Private Inverted-Index Lexical Lookup for Augmented Retrievalcited by 0
  33. 2026TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Trainingcited by 0
  34. 2026Dataset Signatures in Human-LLM Interactions and User Modelingcited by 0
  35. 2026Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agentscited by 0