mlsanityData to 14 Oct 2026

"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents

arXiv 2309.11653 ↗First posted 20 Sep 2023cs.HC cs.AI cs.CRcited by 60

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The widespread use of Large Language Model (LLM)-based conversational agents (CAs), especially in high-stakes domains, raises many privacy concerns. Building ethical LLM-based CAs that respect user privacy requires an in-depth understanding of the privacy risks that concern users the most. However, existing research, primarily model-centered, does not provide insight into users' perspectives. To bridge this gap, we analyzed sensitive disclosures in real-world ChatGPT conversations and conducted semi-structured interviews with 19 LLM-based CA users. We found that users are constantly faced with trade-offs between privacy, utility, and convenience when using LLM-based CAs. However, users' erroneous mental models and the dark patterns in system design limited their awareness and comprehension of the privacy risks. Additionally, the human-like interactions encouraged more sensitive disclosures, which complicated users' ability to navigate the trade-offs. We discuss practical design guidelines and the needs for paradigm shifts to protect the privacy of LLM-based CA users.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 89 references that are in the map. First posted on arXiv 20 Sep 2023.

Builds on

25 of 89 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2020Extracting Training Data from Large Language Modelscited by 2,189
  5. 2021Ethical and social risks of harm from Language Modelscited by 966
  6. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  7. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
  8. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  9. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  10. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  11. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  12. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
13 more
  1. 2023The False Promise of Imitating Proprietary LLMscited by 263
  2. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 214
  3. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  4. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  5. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  6. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
  7. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 51
  8. 2022Differentially Private Decoding in Large Language Modelscited by 34
  9. 2021An Efficient DP-SGD Mechanism for Large Scale NLP Modelscited by 27
  10. 2023Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluationcited by 20
  11. 2023Improved Trust in Human-Robot Collaboration with ChatGPTcited by 18
  12. 2023Several categories of Large Language Models (LLMs): A Short Surveycited by 7
  13. 2019Building chatbots from large scale domain-specific knowledge bases: challenges and opportunitiescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 25; 2026: 19.

60 papers.

Sort:

By area:

  1. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 152
  2. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  3. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  4. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
  5. 2024Towards interactive evaluations for interaction harms in human-AI systemscited by 29
  6. 2025DarkBench: Benchmarking Dark Patterns in Large Language Modelscited by 28
  7. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  8. 2025Position: Privacy Is Not Just Memorization!cited by 17
  9. 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
  10. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  11. 2024Embedding Large Language Models into Extended Reality: Opportunities and Challenges for Inclusion, Engagement, and Privacycited by 14
  12. 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agentcited by 14
38 more
  1. 2024Characterizing and modeling harms from interactions with design patterns in AI interfacescited by 11
  2. 2025Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agentscited by 11
  3. 2025Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Modelscited by 11
  4. 2024Human-Centered Privacy Research in the Age of Large Language Modelscited by 9
  5. 2024Secret Use of Large Language Model (LLM)cited by 9
  6. 2025Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Taskscited by 9
  7. 2024'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMscited by 8
  8. 2024Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recoverycited by 8
  9. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  10. 2024Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metriccited by 6
  11. 2024DAWN: Designing Distributed Agents in a Worldwide Networkcited by 6
  12. 2025The Impact of a Chatbot's Ephemerality-Framing on Self-Disclosure Perceptionscited by 6
  13. 2024Do Large Language Models Possess Sensitive to Sentiment?cited by 3
  14. 2025It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharingcited by 3
  15. 2026Hear You in Silence: Designing for Active Listening in Human Interaction with Conversational Agents Using Context-Aware Pacingcited by 3
  16. 2024Learned, Lagged, LLM-splained: LLM Responses to End User Security Questionscited by 2
  17. 2025Toward Knowledge-Guided AI for Inverse Design in Manufacturing: A Perspective on Domain, Physics, and Human-AI Synergycited by 2
  18. 2025ISACL: Internal State Analyzer for Copyrighted Training Data Leakagecited by 2
  19. 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inferencecited by 2
  20. 2025Can I Trust This Chatbot? Assessing User Privacy in AI-Healthcare Chatbot Applicationscited by 2
  21. 2025Who's Asking? Simulating Role-Based Questions for Conversational AI Evaluationcited by 2
  22. 2025EQ-Negotiator: Dynamic Emotional Personas Empower Small Language Models for Edge-Deployable Credit Negotiationcited by 2
  23. 2026Privacy in Human-AI Romantic Relationships: Concerns, Boundaries, and Agencycited by 2
  24. 2024A Risk Taxonomy and Reflection Tool for Large Language Model Adoption in Public Healthcited by 1
  25. 2025Privacy Meets Explainability: Managing Confidential Data and Transparency Policies in LLM-Empowered Sciencecited by 1
  26. 2025Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agentscited by 1
  27. 2025Black Box Absorption: LLMs Undermining Innovative Ideascited by 1
  28. 2025User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarioscited by 1
  29. 2026Understanding Users' Privacy Reasoning and Behaviors During Chatbot Use to Support Meaningful Agency in Privacycited by 1
  30. 2026GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistantcited by 1
  31. 2026Human-Centred LLM Privacy Audits: Findings and Frictionscited by 1
  32. 2026Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wildcited by 1
  33. 2026Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMscited by 1
  34. 2025When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTscited by 0
  35. 2025One Size Fits All? A Modular Adaptive Sanitization Kit (MASK) for Customizable Privacy-Preserving Phone Scam Detectioncited by 0
  36. 2025Exploring User Acceptance and Concerns toward LLM-powered Conversational Agents in Immersive Extended Realitycited by 0
  37. 2026From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Managementcited by 0
  38. 2026"Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systemscited by 0

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Privacy Control in Conversational LLM Platforms: A Walkthrough Study
  2. 2025Understanding Privacy Norms Around LLM-Based Chatbots: A Contextual Integrity Perspective
  3. 2024The Ethics of Interaction: Mitigating Security Threats in LLMs
  4. 2025Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents
  5. 2024Empowering Users in Digital Privacy Management through Interactive LLM-Based Agents
  6. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents
  7. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action
  8. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents