# AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

arXiv: https://arxiv.org/abs/2603.19574 · HTML: https://mlsanity.com/paper/2603.19574/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 88 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Builds on (most cited first, 28 of 28)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis](https://mlsanity.com/paper/2402.13494/) (2024)
- [Large Language Models in Mental Health Care: a Scoping Review](https://mlsanity.com/paper/2401.02984/) (2024)
- [CounselBench: A Large-Scale Expert Evaluation and Adversarial Benchmarking of Large Language Models in Mental Health Question Answering](https://mlsanity.com/paper/2506.08584/) (2025)
- [Can LLM "Self-report"?: Evaluating the Validity of Self-report Scales in Measuring Personality Design in LLM-based Chatbots](https://mlsanity.com/paper/2412.00207/) (2024)
- [AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression](https://mlsanity.com/paper/2504.18932/) (2025)
- [Mapping Caregiver Needs to AI Chatbot Design: Strengths and Gaps in Mental Health Support for Alzheimer's and Dementia Caregivers](https://mlsanity.com/paper/2506.15047/) (2025)
- [Mental Health Coping Stories on Social Media: A Causal-Inference Study of Papageno Effect](https://mlsanity.com/paper/2302.09885/) (2023)
- [The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models](https://mlsanity.com/paper/2509.10970/) (2025)
- [RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions](https://mlsanity.com/paper/2601.13235/) (2026)
- [Mental Health Impacts of AI Companions: Triangulating Social Media Quasi-Experiments, User Perspectives, and Relational Theory](https://mlsanity.com/paper/2509.22505/) (2025)
- [The ShareLM Collection and Plugin: Contributing Human-Model Chats for the Benefit of the Community](https://mlsanity.com/paper/2408.08291/) (2024)
- [PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Support](https://mlsanity.com/paper/2601.12754/) (2026)
- [Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries](https://mlsanity.com/paper/2504.09271/) (2025)
- [Interpretable Depression Detection from Social Media Text Using LLM-Derived Embeddings](https://mlsanity.com/paper/2506.06616/) (2025)
- [Interpersonal Theory of Suicide as a Lens to Examine Suicidal Ideation in Online Spaces](https://mlsanity.com/paper/2504.13277/) (2025)
- [Eliciting Behaviors in Multi-Turn Conversations](https://mlsanity.com/paper/2512.23701/) (2025)
- [Detecting Early and Implicit Suicidal Ideation via Longitudinal and Information Environment Signals on Social Media](https://mlsanity.com/paper/2510.14889/) (2025)

## Built on by (most cited first, 5 of 5)

- [Dissociating the Internal Representations of Sycophancy in LLMs](https://mlsanity.com/paper/2607.07003/) (2026)
- [Lost in Delusion: Examining LLM Safety Under User Delusions and Distress](https://mlsanity.com/paper/2606.00975/) (2026)
- [Toxic HallucinAItions: Perturbing Prompts and Tracing LLM Circuits](https://mlsanity.com/paper/2605.30913/) (2026)
- [TUX: Measuring Human--AI Tacit Understanding](https://mlsanity.com/paper/2605.30930/) (2026)
- [PeerPen: AI-Assisted Writing for Online Mental Health Peer Support](https://mlsanity.com/paper/2609.14886/) (2026)

## Abstract (arXiv)

Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking---a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.