# Right Words, Wrong Moment: A Clinician-Grounded Analysis of Distress in 19,930 Conversations between Young People and ChatGPT

arXiv: https://arxiv.org/abs/2609.35953 · HTML: https://mlsanity.com/paper/2609.35953/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 90 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Preference Ranking Optimization for Human Alignment](https://mlsanity.com/paper/2306.17492/) (2023)
- [SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations](https://mlsanity.com/paper/2311.00273/) (2023)
- [Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support](https://mlsanity.com/paper/2307.15810/) (2023)
- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild](https://mlsanity.com/paper/2407.11438/) (2024)
- [Investigating Affective Use and Emotional Well-being on ChatGPT](https://mlsanity.com/paper/2504.03888/) (2025)
- [Navigating Rifts in Human-LLM Grounding: Study and Benchmark](https://mlsanity.com/paper/2503.13975/) (2025)
- [Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation](https://mlsanity.com/paper/2406.11852/) (2024)
- [Characterizing Delusional Spirals through Human-LLM Chat Logs](https://mlsanity.com/paper/2603.16567/) (2026)
- [The Role of AI in Peer Support for Young People: A Study of Preferences for Human- and AI-Generated Responses](https://mlsanity.com/paper/2405.02711/) (2024)
- [Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives](https://mlsanity.com/paper/2507.15783/) (2025)
- [Relational Norms for Human-AI Cooperation](https://mlsanity.com/paper/2502.12102/) (2025)
- [Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recovery](https://mlsanity.com/paper/2412.11656/) (2024)
- ["I've talked to ChatGPT about my issues last night.": Examining Mental Health Conversations with Large Language Models through Reddit Analysis](https://mlsanity.com/paper/2504.20320/) (2025)
- [Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs](https://mlsanity.com/paper/2509.24857/) (2025)
- [Principles of Safe AI Companions for Youth: Parent and Expert Perspectives](https://mlsanity.com/paper/2510.11185/) (2025)
- [Workshop Paper: An empirical study to understand how students use ChatGPT for writing essays and how it affects their ownership](https://mlsanity.com/paper/2405.13890/) (2024)
- [Taking Advice from ChatGPT](https://mlsanity.com/paper/2305.11888/) (2023)

## Abstract (arXiv)

Young people increasingly turn to General-Purpose Conversational Agents (GPCAs), such as ChatGPT, in moments of distress. We examine young adults' (ages 18-25) experiences using ChatGPT. We first collected 19,930 ChatGPT conversations and survey data from 158 young adults. We then selected five example conversations reflecting user distress. Finally, we asked ten clinicians to review those five conversations. We found distressed participants reported greater emotional engagement with ChatGPT and greater behavioral change from using it than their peers. When they turned to ChatGPT in moments of acute distress, ChatGPT was quick to give overly dramatic responses and excessive action-oriented suggestions. Clinicians endorsed ChatGPT's availability and much of its wording, but identified seven process failures, such as prematurely jumping to solutions. We translated clinicians' feedback into design guidelines following three stages: 1) asking about safety, 2) de-escalating intensity to restore emotional regulation, and 3) exploring concerns without agreeing with them.