Playing Games with My Heart: An Evaluation of AI Companion Apps
Not on the map. Why?
Abstract (arXiv)
The use of chatbots for various forms of companionship is growing rapidly, raising a myriad of questions about simulated relationships, emotional dependence, and psychological harm. While major platforms such as ChatGPT, Grok, and Character AI are the subject of a growing body of research and legal inquiries, apps explicitly built for simulating intimate interpersonal relationships remain under-explored. In this work, we evaluate the five most popular AI companion mobile applications for factors that encourage parasocial interaction and may manipulate users. We do this by manually annotating the user experience each offers. Specifically, we systematically record and quantify design dark patterns, anthropomorphism, stereotypes, erotica, and technical performance issues. We find that all apps contain substantial dark patterns aimed at increasing monetisation and user engagement. Erotica and gamification such as levelling are also prevalent, and although features vary considerably between applications, all apps have highly anthropomorphic design. These findings shed light on the mechanics used to leverage users' simulated relationships. On that basis, we put forward recommendations for regulators to strengthen consumer protection in this rapidly emerging market. Content warning: This article contains objectifying images of women, erotic images, textual references to incest, and other potentially sensitive, offensive, and distressing text.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 102 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
20 of 102 references.
References · 20
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
- 2023SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Modelscited by 66
- 2024The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationshipscited by 61
- 2024Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attributioncited by 39
- 2025Investigating Affective Use and Emotional Well-being on ChatGPTcited by 36
- 2021Revealing Persona Biases in Dialogue Systemscited by 33
- 2024Stereotype or Personalization? User Identity Biases Chatbot Recommendationscited by 33
- 2023Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Sociallycited by 20
- 2025Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationshipscited by 17
- 2021What Makes a Dark Pattern... Dark? Design Attributes, Normative Considerations, and Measurement Methodscited by 15
- 2025A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologiescited by 14
8 more
- 2025Humanlike AI Design Increases Anthropomorphism but Yields Divergent Outcomes on Engagement and Trust Globallycited by 6
- 2023Defending Against the Dark Arts: Recognising Dark Patterns in Social Mediacited by 5
- 2025Detecting Linguistic Indicators for Stereotype Assessment with Large Language Modelscited by 5
- 2025Mapping the Parasocial AI Market: User Trends, Engagement and Riskscited by 5
- 2025AI Will Always Love You: Studying Implicit Biases in Romantic AI Companionscited by 3
- 2026Examining Risks Through a Characterization of the AI Companion Application Ecosystem: A Stratified Sample from the Apple App Store and Google Play Storecited by 2
- 2024The Engagement-Prolonging Designs Teens Encounter on Very Large Online Platformscited by 1
- 2026Terms of (Ab)Use: An Analysis of GenAI Servicescited by 1
Built on by
Not cited yet.