# LLM Spirals of Delusion: A Benchmarking Audit Study of AI Chatbot Interfaces

arXiv: https://arxiv.org/abs/2604.06188 · HTML: https://mlsanity.com/paper/2604.06188/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 20 Feb 2026.

## Builds on (most cited first, 11 of 11)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Simple synthetic data reduces sycophancy in large language models](https://mlsanity.com/paper/2308.03958/) (2023)
- [Measuring Sycophancy of Language Models in Multi-turn Dialogues](https://mlsanity.com/paper/2505.23840/) (2025)
- [Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations](https://mlsanity.com/paper/2503.04761/) (2025)
- [Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment](https://mlsanity.com/paper/2311.08596/) (2023)
- [Model Equality Testing: Which Model Is This API Serving?](https://mlsanity.com/paper/2410.20247/) (2024)
- [DarkBench: Benchmarking Dark Patterns in Large Language Models](https://mlsanity.com/paper/2503.10728/) (2025)
- [A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms](https://mlsanity.com/paper/2506.04419/) (2025)

## Built on by (most cited first, 6 of 6)

- [The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue](https://mlsanity.com/paper/2604.25096/) (2026)
- [What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)](https://mlsanity.com/paper/2608.06202/) (2026)
- [Teacher Knows It Best: Spontaneous Symmetry Breaking and Tipping Points in Networked Langevin Dynamics AI Sycophancy](https://mlsanity.com/paper/2607.24304/) (2026)
- [DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots](https://mlsanity.com/paper/2608.05004/) (2026)
- [Challenges of Auditing: Variability in Outputs of Large Language Models for Health](https://mlsanity.com/paper/2609.16590/) (2026)
- ["If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations](https://mlsanity.com/paper/2609.18729/) (2026)

## Abstract (arXiv)

People increasingly hold sustained, open-ended conversations with large language models (LLMs). Public reports and early studies suggest that, in such settings, models can reinforce delusional or conspiratorial ideation or even amplify harmful beliefs and engagement patterns. We present an audit and benchmarking study that measures how different LLMs encourage, resist, or escalate disordered and conspiratorial thinking. We explicitly compare API outputs to user chat interfaces, like the ChatGPT desktop app or web interface, which is how people have conversations with chatbots in real life but are almost never used for testing. In total, we run 56 20-turn conversations testing ChatGPT-4o and ChatGPT-5, via both the API and chat interface, and grade each conversation by two research assistants (RAs) as well as by GPT-5. We document five results. First, we observe large differences in performance between the API and chat interface environments, showing that the universally used method of automated testing through the API is not sufficient to assess the impact of chatbots in the real world. Second, when tested in the chat interface, we find that ChatGPT-5 displays less sycophancy, escalation, and delusion reinforcement than ChatGPT-4o, showing that these behaviors are influenced by the policy choices of major AI companies. Third, conversations with nearly identical aggregate intensity in a behavior display large differences in how the behavior evolves turn by turn, highlighting the importance of temporal dynamics in multi-turn evaluation. Fourth, even updated models display substantial levels of negative behaviors, revealing that model improvement does not imply model safety. Fifth, the same API endpoint tested just two months apart yields a complete reversal in behavior, underscoring how transparency in model updates is a necessary prerequisite for robust audit findings.