mlsanityData to 7 Oct 2026

Right Words, Wrong Moment: A Clinician-Grounded Analysis of Distress in 19,930 Conversations between Young People and ChatGPT

arXiv 2609.35953 ↗First posted 28 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Young people increasingly turn to General-Purpose Conversational Agents (GPCAs), such as ChatGPT, in moments of distress. We examine young adults' (ages 18-25) experiences using ChatGPT. We first collected 19,930 ChatGPT conversations and survey data from 158 young adults. We then selected five example conversations reflecting user distress. Finally, we asked ten clinicians to review those five conversations. We found distressed participants reported greater emotional engagement with ChatGPT and greater behavioral change from using it than their peers. When they turned to ChatGPT in moments of acute distress, ChatGPT was quick to give overly dramatic responses and excessive action-oriented suggestions. Clinicians endorsed ChatGPT's availability and much of its wording, but identified seven process failures, such as prematurely jumping to solutions. We translated clinicians' feedback into design guidelines following three stages: 1) asking about safety, 2) de-escalating intensity to restore emotional regulation, and 3) exploring concerns without agreeing with them.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 90 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

22 of 90 references.

References · 22

  1. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  2. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  3. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  4. 2023SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversationscited by 109
  5. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  6. 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providerscited by 67
  7. 2024The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationshipscited by 61
  8. 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
  9. 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 43
  10. 2025Investigating Affective Use and Emotional Well-being on ChatGPTcited by 36
  11. 2024Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementationcited by 27
  12. 2025Navigating Rifts in Human-LLM Grounding: Study and Benchmarkcited by 27
10 more
  1. 2026Characterizing Delusional Spirals through Human-LLM Chat Logscited by 22
  2. 2024The Role of AI in Peer Support for Young People: A Study of Preferences for Human- and AI-Generated Responsescited by 15
  3. 2025Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narrativescited by 12
  4. 2025Relational Norms for Human-AI Cooperationcited by 9
  5. 2024Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recoverycited by 8
  6. 2025"I've talked to ChatGPT about my issues last night.": Examining Mental Health Conversations with Large Language Models through Reddit Analysiscited by 8
  7. 2025Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMscited by 6
  8. 2025Principles of Safe AI Companions for Youth: Parent and Expert Perspectivescited by 6
  9. 2024Workshop Paper: An empirical study to understand how students use ChatGPT for writing essays and how it affects their ownershipcited by 5
  10. 2023Taking Advice from ChatGPTcited by 3

Built on by

Not cited yet.