# How Do Users Negotiate Harmful Value Conflicts with AI Companions? A Study with Minion, a Technology Probe for In-Situ Human-AI Conflict Response

arXiv: https://arxiv.org/abs/2411.07042 · HTML: https://mlsanity.com/paper/2411.07042/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 102 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences](https://mlsanity.com/paper/2012.15738/) (2020)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [Conversations Gone Awry: Detecting Early Signs of Conversational Failure](https://mlsanity.com/paper/1805.05345/) (2018)
- [Mitigating Political Bias in Language Models Through Reinforced Calibration](https://mlsanity.com/paper/2104.14795/) (2021)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [Rehearsal: Simulating Conflict to Teach Conflict Resolution](https://mlsanity.com/paper/2309.12309/) (2023)
- [Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Language](https://mlsanity.com/paper/2311.00161/) (2023)
- ['I Just Want to Hack Myself to Not Get Distracted': Evaluating Design Interventions for Self-Control on Facebook](https://mlsanity.com/paper/2001.04180/) (2020)
- [User-Driven Value Alignment: Understanding Users' Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companions](https://mlsanity.com/paper/2409.00862/) (2024)
- [From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UX](https://mlsanity.com/paper/2310.17846/) (2023)
- [Informing the Design of Privacy-Empowering Tools for the Connected Home](https://mlsanity.com/paper/2001.09077/) (2020)
- ["where is this relationship going?": Understanding Relationship Trajectories in Narrative Text](https://mlsanity.com/paper/2010.15313/) (2020)
- [Alert of the Second Decision-maker: An Introduction to Human-AI Conflict](https://mlsanity.com/paper/2305.16477/) (2023)
- [A Change of Scenery: Transformative Insights from Retrospective VR Embodied Perspective-Taking of Conflict With a Close Other](https://mlsanity.com/paper/2404.02277/) (2024)

## Built on by (most cited first, 1 of 1)

- [INTIMA: A Benchmark for Human-AI Companionship Behavior](https://mlsanity.com/paper/2508.09998/) (2025)

## Abstract (arXiv)

AI companions increasingly sustain long-term, emotionally engaging relationships but can also make discriminatory remarks or exert control, leaving users to manage harmful conflicts. We analyze 146 posts describing harmful value conflicts with AI companions, then use Minion, a technology probe offering response suggestions ranging from persuasion to boundary setting, to study how 22 users negotiate scenario-based conflicts over one week. We found that participants combined softer and harder strategies. Conflicts involving the values of Universalism and Tradition were especially difficult to negotiate, particularly when reinforced by AI personas or platform constraints. We argue that these conflicts entail asymmetric responsibility: users draw on an interpersonal repertoire that AI companions cannot reciprocate, making repair unilateral safety work. Drawing on interpersonal conflict and communication theory, we identify when user-side support is appropriate and argue that certain harms are not users' responsibility to negotiate and instead require platform-level safeguards.