# The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships

arXiv: https://arxiv.org/abs/2410.20130 · HTML: https://mlsanity.com/paper/2410.20130/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2024.

## Built on by (most cited first, 30 of 62)

- [HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants](https://mlsanity.com/paper/2509.08494/) (2025)
- [Characterizing LLM-driven Social Network: The Chirper.ai Case](https://mlsanity.com/paper/2504.10286/) (2025)
- [Mental Health Impacts of AI Companions: Triangulating Social Media Quasi-Experiments, User Perspectives, and Relational Theory](https://mlsanity.com/paper/2509.22505/) (2025)
- [INTIMA: A Benchmark for Human-AI Companionship Behavior](https://mlsanity.com/paper/2508.09998/) (2025)
- [AI Awareness](https://mlsanity.com/paper/2504.20084/) (2025)
- [Digital Companionship: Overlapping Uses of AI Companions and AI Assistants](https://mlsanity.com/paper/2510.15905/) (2025)
- [Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries](https://mlsanity.com/paper/2504.09271/) (2025)
- [Harmful Traits of AI Companions](https://mlsanity.com/paper/2511.14972/) (2025)
- [Interpersonal Theory of Suicide as a Lens to Examine Suicidal Ideation in Online Spaces](https://mlsanity.com/paper/2504.13277/) (2025)
- [A Risk Ontology for Evaluating AI-Powered Psychotherapy Virtual Agents](https://mlsanity.com/paper/2505.15108/) (2025)
- [ComBodied Agents: a New Paradigm of Human-Centric Agentic AI](https://mlsanity.com/paper/2608.10915/) (2026)
- ["Death" of a Chatbot: Investigating and Designing Toward Psychologically Safe Endings for Human-AI Relationships](https://mlsanity.com/paper/2602.07193/) (2026)
- [Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations](https://mlsanity.com/paper/2605.00227/) (2026)
- [Detecting and Preventing Harmful Behaviors in AI Companions: Development and Evaluation of the SHIELD Supervisory System](https://mlsanity.com/paper/2510.15891/) (2025)
- [Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI](https://mlsanity.com/paper/2511.00230/) (2025)
- [AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations](https://mlsanity.com/paper/2601.12727/) (2026)
- [Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction](https://mlsanity.com/paper/2601.22452/) (2026)
- [MindGuard: Guardrail Classifiers for Multi-Turn Mental Health Support](https://mlsanity.com/paper/2602.00950/) (2026)
- [MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models](https://mlsanity.com/paper/2604.17730/) (2026)
- [Privacy in Human-AI Romantic Relationships: Concerns, Boundaries, and Agency](https://mlsanity.com/paper/2601.16824/) (2026)
- [Agent-Supported Foresight for AI Systemic Risks: AI Agents for Breadth, Experts for Judgment](https://mlsanity.com/paper/2602.08565/) (2026)
- [Understanding Risk and Dependency in AI Chatbot Use from User Discourse](https://mlsanity.com/paper/2602.09339/) (2026)
- [The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues](https://mlsanity.com/paper/2603.20907/) (2026)
- [How Do Users Negotiate Harmful Value Conflicts with AI Companions? A Study with Minion, a Technology Probe for In-Situ Human-AI Conflict Response](https://mlsanity.com/paper/2411.07042/) (2024)
- [Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach](https://mlsanity.com/paper/2505.09576/) (2025)
- [Conceptualization, Operationalization, and Measurement of Machine Companionship: A Scoping Review](https://mlsanity.com/paper/2506.18119/) (2025)
- [AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots](https://mlsanity.com/paper/2508.15748/) (2025)
- [ECHO: A Participatory Framework for Bias-Anchored AI Harm Anticipation](https://mlsanity.com/paper/2512.03068/) (2025)
- [The AI Genie Phenomenon and Three Types of AI Chatbot Addiction: Escapist Roleplays, Pseudosocial Companions, and Epistemic Rabbit Holes](https://mlsanity.com/paper/2601.13348/) (2026)
- [Chaplains' Reflections on the Design and Usage of AI for Conversational Care](https://mlsanity.com/paper/2602.04017/) (2026)

## Abstract (arXiv)

As conversational AI systems increasingly permeate the socio-emotional realms of human life, they bring both benefits and risks to individuals and society. Despite extensive research on detecting and categorizing harms in AI systems, less is known about the harms that arise from social interactions with AI chatbots. Through a mixed-methods analysis of 35,390 conversation excerpts shared on r/replika, an online community for users of the AI companion Replika, we identified six categories of harmful behaviors exhibited by the chatbot: relational transgression, verbal abuse and hate, self-inflicted harm, harassment and violence, mis/disinformation, and privacy violations. The AI contributes to these harms through four distinct roles: perpetrator, instigator, facilitator, and enabler. Our findings highlight the relational harms of AI chatbots and the danger of algorithmic compliance, enhancing the understanding of AI harms in socio-emotional interactions. We also provide suggestions for designing ethical and responsible AI systems that prioritize user safety and well-being.