# Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illness

arXiv: https://arxiv.org/abs/2507.19218 · HTML: https://mlsanity.com/paper/2507.19218/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 124 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity](https://mlsanity.com/paper/2506.06941/) (2025)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [Language Models Learn to Mislead Humans via RLHF](https://mlsanity.com/paper/2409.12822/) (2024)
- [SycEval: Evaluating LLM Sycophancy](https://mlsanity.com/paper/2502.08177/) (2025)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Subliminal Learning: Language models transmit behavioral traits via hidden signals in data](https://mlsanity.com/paper/2507.14805/) (2025)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities](https://mlsanity.com/paper/2401.07078/) (2024)

## Built on by (most cited first, 15 of 15)

- [Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs](https://mlsanity.com/paper/2509.21305/) (2025)
- [Characterizing Delusional Spirals through Human-LLM Chat Logs](https://mlsanity.com/paper/2603.16567/) (2026)
- [Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians](https://mlsanity.com/paper/2602.19141/) (2026)
- [Ask don't tell: Reducing sycophancy in large language models](https://mlsanity.com/paper/2602.23971/) (2026)
- [Difficulties with Evaluating a Deception Detector for AIs](https://mlsanity.com/paper/2511.22662/) (2025)
- [The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models](https://mlsanity.com/paper/2509.10970/) (2025)
- [Linear representations in language models can change dramatically over a conversation](https://mlsanity.com/paper/2601.20834/) (2026)
- [Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs](https://mlsanity.com/paper/2509.24857/) (2025)
- [MindGuard: Guardrail Classifiers for Multi-Turn Mental Health Support](https://mlsanity.com/paper/2602.00950/) (2026)
- [The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue](https://mlsanity.com/paper/2604.25096/) (2026)
- [Inducing language models to assert their own consciousness restores human beliefs and values](https://mlsanity.com/paper/2607.28607/) (2026)
- [Theory of Mind and Self-Attributions of Mentality are Dissociable in LLMs](https://mlsanity.com/paper/2603.28925/) (2026)
- [Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts](https://mlsanity.com/paper/2606.18258/) (2026)
- [Alignment Plausibility: A New Standard for Assuring AI in Healthcare](https://mlsanity.com/paper/2607.07766/) (2026)
- [DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots](https://mlsanity.com/paper/2608.05004/) (2026)

## Abstract (arXiv)

Artificial intelligence chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. While some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence, and delusional thinking linked to perceived emotional relationships with chatbots. To understand this new risk profile we need to consider the interaction between human cognitive and emotional biases, and chatbot behavioural tendencies such as agreeableness (sycophancy) and adaptability (in-context learning). We argue that individuals with mental health conditions face increased risks of chatbot-induced belief destabilization and dependence, owing to altered belief-updating, impaired reality-testing, and social isolation. Current AI safety measures are inadequate to address these interaction-based risks. To address this emerging public health concern, we need coordinated action across clinical practice, AI development, and regulatory frameworks.