mlsanityData to 7 Oct 2026

How Do Users Negotiate Harmful Value Conflicts with AI Companions? A Study with Minion, a Technology Probe for In-Situ Human-AI Conflict Response

arXiv 2411.07042 ↗First posted 11 Nov 2024cs.HC cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

AI companions increasingly sustain long-term, emotionally engaging relationships but can also make discriminatory remarks or exert control, leaving users to manage harmful conflicts. We analyze 146 posts describing harmful value conflicts with AI companions, then use Minion, a technology probe offering response suggestions ranging from persuasion to boundary setting, to study how 22 users negotiate scenario-based conflicts over one week. We found that participants combined softer and harder strategies. Conflicts involving the values of Universalism and Tradition were especially difficult to negotiate, particularly when reinforced by AI personas or platform constraints. We argue that these conflicts entail asymmetric responsibility: users draw on an interpersonal repertoire that AI companions cannot reciprocate, making repair unilateral safety work. Drawing on interpersonal conflict and communication theory, we identify when user-side support is appropriate and argue that certain harms are not users' responsibility to negotiate and instead require platform-level safeguards.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 102 references that are in the map. First posted on arXiv 11 Nov 2024.

Builds on

20 of 102 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  3. 2020Aligning AI With Shared Human Valuescited by 706
  4. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  5. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  6. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  7. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  8. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  9. 2018Conversations Gone Awry: Detecting Early Signs of Conversational Failurecited by 73
  10. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  11. 2024The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationshipscited by 61
  12. 2023Rehearsal: Simulating Conflict to Teach Conflict Resolutioncited by 54
8 more
  1. 2023Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Languagecited by 22
  2. 2020'I Just Want to Hack Myself to Not Get Distracted': Evaluating Design Interventions for Self-Control on Facebookcited by 17
  3. 2024User-Driven Value Alignment: Understanding Users' Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companionscited by 14
  4. 2023From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UXcited by 5
  5. 2020Informing the Design of Privacy-Empowering Tools for the Connected Homecited by 3
  6. 2020"where is this relationship going?": Understanding Relationship Trajectories in Narrative Textcited by 3
  7. 2023Alert of the Second Decision-maker: An Introduction to Human-AI Conflictcited by 1
  8. 2024A Change of Scenery: Transformative Insights from Retrospective VR Embodied Perspective-Taking of Conflict With a Close Othercited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.