Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Health
Not on the map. Why?
Abstract (arXiv)
Individuals are increasingly relying on large language model (LLM)-enabled conversational agents for emotional support. While prior research has examined privacy and security issues in chatbots specifically designed for mental health purposes, these chatbots are overwhelmingly "rule-based" offerings that do not leverage generative AI. Little empirical research currently measures users' privacy and security concerns, attitudes, and expectations when using general-purpose LLM-enabled chatbots to manage and improve mental health. Through 21 semi-structured interviews with U.S. participants, we identified critical misconceptions and a general lack of risk awareness. Participants conflated the human-like empathy exhibited by LLMs with human-like accountability and mistakenly believed that their interactions with these chatbots were safeguarded by the same regulations (e.g., HIPAA) as disclosures with a licensed therapist. We introduce the concept of "intangible vulnerability," where emotional or psychological disclosures are undervalued compared to more tangible forms of information (e.g., financial or location-based data). To address this, we propose recommendations to safeguard user mental health disclosures with general-purpose LLM-enabled chatbots more effectively.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 13 of its 124 references that are in the map. First posted on arXiv 14 Jul 2025.
Builds on
13 of 124 references.
References · 13
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
- 2023Cognitive Reframing of Negative Thoughts through Human-Language Model Interactioncited by 77
- 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
- 2024The opportunities and risks of large language models in mental healthcited by 54
- 2023MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journalingcited by 50
- 2023Can Language Models be Instructed to Protect Personal Information?cited by 46
- 2023Benefits and Harms of Large Language Models in Digital Mental Healthcited by 40
- 2022Inducing Positive Perspectives with Text Reframingcited by 31
- 2024The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Supportcited by 24
- 2023Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Supportcited by 17
1 more
Built on by
Citing papers per year: 2025: 9; 2026: 4.
13 papers.
Sort:
By area:
- 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacycited by 7
- 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
- 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Healthcited by 5
- 2025Can LLMs Address Mental Health Questions? A Comparison with Human Therapistscited by 4
- 2025It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharingcited by 3
- 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inferencecited by 2
- 2025A Longitudinal Measurement of Privacy Policy Evolution for Large Language Modelscited by 2
- 2025MEASER: Malware embedding attacks on open-source LLMscited by 0
- 2025Mutual Wanting in Human--AI Interaction: Empirical Evidence from Large-Scale Analysis of GPT Model Transitionscited by 0
- 2026Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health Informationcited by 0
- 2026Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AIcited by 0
- 2026Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layerscited by 0
- 2026Cloze: An Open Research Platform for Studying Human-AI Conversations in Mental Health Contexts13 Jun 2026
- 2026Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layers22 Apr 2026
- 2026Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI19 Apr 2026
- 2026Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health Information10 Mar 2026
- 2025A Longitudinal Measurement of Privacy Policy Evolution for Large Language Models24 Nov 2025
- 2025Mutual Wanting in Human--AI Interaction: Empirical Evidence from Large-Scale Analysis of GPT Model Transitions27 Oct 2025
- 2025MEASER: Malware embedding attacks on open-source LLMs12 Oct 2025
- 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacy5 Oct 2025
- 2025It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing26 Sep 2025
- 2025Beyond Data Privacy: New Privacy Risks for Large Language Models16 Sep 2025
- 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Health16 Sep 2025
- 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inference15 Sep 2025
1 more
- 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
- 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Healthcited by 5
- 2025Can LLMs Address Mental Health Questions? A Comparison with Human Therapistscited by 4
- 2025It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharingcited by 3
- 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inferencecited by 2
- 2025MEASER: Malware embedding attacks on open-source LLMscited by 0
- 2025Mutual Wanting in Human--AI Interaction: Empirical Evidence from Large-Scale Analysis of GPT Model Transitionscited by 0
- 2026Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health Informationcited by 0
- 2026Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AIcited by 0
- 2026Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layerscited by 0
- 2026Cloze: An Open Research Platform for Studying Human-AI Conversations in Mental Health Contextscited by 0
- 2026Cloze: An Open Research Platform for Studying Human-AI Conversations in Mental Health Contexts13 Jun 2026
- 2026Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layers22 Apr 2026
- 2026Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI19 Apr 2026
- 2026Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health Information10 Mar 2026
- 2025Mutual Wanting in Human--AI Interaction: Empirical Evidence from Large-Scale Analysis of GPT Model Transitions27 Oct 2025
- 2025MEASER: Malware embedding attacks on open-source LLMs12 Oct 2025
- 2025It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing26 Sep 2025
- 2025Beyond Data Privacy: New Privacy Risks for Large Language Models16 Sep 2025
- 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Health16 Sep 2025
- 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inference15 Sep 2025
- 2025Can LLMs Address Mental Health Questions? A Comparison with Human Therapists15 Sep 2025