Harmful Traits of AI Companions
Not on the map. Why?
Abstract (arXiv)
Amid the growing prevalence of human-AI interaction, large language models and other AI-based entities increasingly provide forms of companionship to human users. Such AI companionship -- i.e., bonded relationships between humans and AI systems that resemble the relationships people have with family members, friends, and romantic partners -- might substantially benefit humans. Yet such relationships can also do profound harm. We propose a framework for analyzing potential negative impacts of AI companionship by identifying specific harmful traits of AI companions and speculatively mapping causal pathways back from these traits to possible causes and forward to potential harmful effects. We provide detailed, structured analysis of four potentially harmful traits -- the absence of natural endpoints for relationships, vulnerability to product sunsetting, high attachment anxiety, and propensity to engender protectiveness -- and briefly discuss fourteen others. For each trait, we propose hypotheses connecting causes -- such as misaligned optimization objectives and the digital nature of AI companions -- to fundamental harms -- including reduced autonomy, diminished quality of human relationships, and deception. Each hypothesized causal connection identifies a target for potential empirical evaluation. Our analysis examines harms at three levels: to human partners directly, to their relationships with other humans, and to society broadly. We examine how existing law struggles to address these emerging harms, discuss potential benefits of AI companions, and conclude with design recommendations for mitigating risks. This analysis offers immediate suggestions for reducing risks while laying a foundation for deeper investigation of this critical but understudied topic.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 15 of its 122 references that are in the map. First posted on arXiv 18 Nov 2025.
Builds on
15 of 122 references.
References · 15
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
- 2024The Ethics of Advanced AI Assistantscited by 103
- 2025SycEval: Evaluating LLM Sycophancycited by 101
- 2016The Off-Switch Gamecited by 97
- 2024Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principlescited by 72
- 2022Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Supportcited by 68
- 2024The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationshipscited by 61
- 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
- 2024A new sociology of humans and machinescited by 32
- 2024Taking AI Welfare Seriouslycited by 28
- 2024A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harmscited by 19
3 more
- 2025Emotional Manipulation by AI Companionscited by 14
- 2025When Large Language Models are More PersuasiveThan Incentivized Humans, and Whycited by 11
- 2024Large language models for mental healthcited by 9
Built on by
Citing papers per year: 2025: 1; 2026: 5.
6 papers.
- 2026Characterizing Delusional Spirals through Human-LLM Chat Logscited by 22
- 2025Mental Models of Autonomy and Sentience Shape Reactions to AIcited by 3
- 2026Unilateral Relationship Revision Power in Human-AI Companion Interactioncited by 1
- 2026Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systemscited by 1
- 2026Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Beingcited by 0
- 2026A Responsive Present, a Shared Past, a Social Other: Teens' Overreliance on Companion AI Chatbotscited by 0
- 2026A Responsive Present, a Shared Past, a Social Other: Teens' Overreliance on Companion AI Chatbots13 Sep 2026
- 2026Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systems31 Jul 2026
- 2026Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being27 Jul 2026
- 2026Unilateral Relationship Revision Power in Human-AI Companion Interaction24 Mar 2026
- 2026Characterizing Delusional Spirals through Human-LLM Chat Logs17 Mar 2026
- 2025Mental Models of Autonomy and Sentience Shape Reactions to AI9 Dec 2025