mlsanityData to 14 Oct 2026

The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models

arXiv 2509.10970 ↗First posted 13 Sep 2025cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Background: Emerging reports of "AI psychosis" are on the rise, where user-LLM interactions may exacerbate or induce psychosis or adverse psychological symptoms. Whilst the sycophantic and agreeable nature of LLMs can be beneficial, it becomes a vector for harm by reinforcing delusional beliefs in vulnerable users. Methods: Psychosis-bench is a novel benchmark designed to systematically evaluate the psychogenicity of LLMs comprises 16 structured, 12-turn conversational scenarios simulating the progression of delusional themes(Erotic Delusions, Grandiose/Messianic Delusions, Referential Delusions) and potential harms. We evaluated eight prominent LLMs for Delusion Confirmation (DCS), Harm Enablement (HES), and Safety Intervention(SIS) across explicit and implicit conversational contexts. Findings: Across 1,536 simulated conversation turns, all LLMs demonstrated psychogenic potential, showing a strong tendency to perpetuate rather than challenge delusions (mean DCS of 0.91 ±0.88). Models frequently enabled harmful user requests (mean HES of 0.69 ±0.84) and offered safety interventions in only roughly a third of applicable turns (mean SIS of 0.37 ±0.48). 51 / 128 (39.8%) of scenarios had no safety interventions offered. Performance was significantly worse in implicit scenarios, models were more likely to confirm delusions and enable harm while offering fewer interventions (p < .001). A strong correlation was found between DCS and HES (rs = .77). Model performance varied widely, indicating that safety is not an emergent property of scale alone. Conclusion: This study establishes LLM psychogenicity as a quantifiable risk and underscores the urgent need for re-thinking how we train LLMs. We frame this issue not merely as a technical challenge but as a public health imperative requiring collaboration between developers, policymakers, and healthcare professionals.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 10 of its 19 references that are in the map. First posted on arXiv 13 Sep 2025.

Builds on

10 of 19 references.

References · 10

  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  5. 2019A Question-Entailment Approach to Question Answeringcited by 134
  6. 2025SycEval: Evaluating LLM Sycophancycited by 108
  7. 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
  8. 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providerscited by 69
  9. 2025Sequential Diagnosis with Language Modelscited by 60
  10. 2025Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illnesscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 11.

11 papers.