# Harmful Traits of AI Companions

arXiv: https://arxiv.org/abs/2511.14972 · HTML: https://mlsanity.com/paper/2511.14972/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 122 references that are in the map.
- First posted on arXiv 18 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [SycEval: Evaluating LLM Sycophancy](https://mlsanity.com/paper/2502.08177/) (2025)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [The Off-Switch Game](https://mlsanity.com/paper/1611.08219/) (2016)
- [Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles](https://mlsanity.com/paper/2407.00870/) (2024)
- [Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support](https://mlsanity.com/paper/2203.15144/) (2022)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [Large Language Models Produce Responses Perceived to be Empathic](https://mlsanity.com/paper/2403.18148/) (2024)
- [A new sociology of humans and machines](https://mlsanity.com/paper/2402.14410/) (2024)
- [Taking AI Welfare Seriously](https://mlsanity.com/paper/2411.00986/) (2024)
- [A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms](https://mlsanity.com/paper/2407.01294/) (2024)
- [Emotional Manipulation by AI Companions](https://mlsanity.com/paper/2508.19258/) (2025)
- [When Large Language Models are More PersuasiveThan Incentivized Humans, and Why](https://mlsanity.com/paper/2505.09662/) (2025)
- [Large language models for mental health](https://mlsanity.com/paper/2411.11880/) (2024)

## Built on by (most cited first, 6 of 6)

- [Characterizing Delusional Spirals through Human-LLM Chat Logs](https://mlsanity.com/paper/2603.16567/) (2026)
- [Mental Models of Autonomy and Sentience Shape Reactions to AI](https://mlsanity.com/paper/2512.09085/) (2025)
- [Unilateral Relationship Revision Power in Human-AI Companion Interaction](https://mlsanity.com/paper/2603.23315/) (2026)
- [Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systems](https://mlsanity.com/paper/2608.00151/) (2026)
- [Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being](https://mlsanity.com/paper/2607.25057/) (2026)
- [A Responsive Present, a Shared Past, a Social Other: Teens' Overreliance on Companion AI Chatbots](https://mlsanity.com/paper/2609.14843/) (2026)

## Abstract (arXiv)

Amid the growing prevalence of human-AI interaction, large language models and other AI-based entities increasingly provide forms of companionship to human users. Such AI companionship -- i.e., bonded relationships between humans and AI systems that resemble the relationships people have with family members, friends, and romantic partners -- might substantially benefit humans. Yet such relationships can also do profound harm. We propose a framework for analyzing potential negative impacts of AI companionship by identifying specific harmful traits of AI companions and speculatively mapping causal pathways back from these traits to possible causes and forward to potential harmful effects. We provide detailed, structured analysis of four potentially harmful traits -- the absence of natural endpoints for relationships, vulnerability to product sunsetting, high attachment anxiety, and propensity to engender protectiveness -- and briefly discuss fourteen others. For each trait, we propose hypotheses connecting causes -- such as misaligned optimization objectives and the digital nature of AI companions -- to fundamental harms -- including reduced autonomy, diminished quality of human relationships, and deception. Each hypothesized causal connection identifies a target for potential empirical evaluation. Our analysis examines harms at three levels: to human partners directly, to their relationships with other humans, and to society broadly. We examine how existing law struggles to address these emerging harms, discuss potential benefits of AI companions, and conclude with design recommendations for mitigating risks. This analysis offers immediate suggestions for reducing risks while laying a foundation for deeper investigation of this critical but understudied topic.