# Why human-AI relationships need socioaffective alignment

arXiv: https://arxiv.org/abs/2502.02528 · HTML: https://mlsanity.com/paper/2502.02528/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 147 references that are in the map.
- First posted on arXiv 4 Feb 2025.

## Builds on (most cited first, 30 of 31)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [Language Models as Agent Models](https://mlsanity.com/paper/2212.01681/) (2022)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [Agent Incentives: A Causal Perspective](https://mlsanity.com/paper/2102.01685/) (2021)
- [Beyond Preferences in AI Alignment](https://mlsanity.com/paper/2408.16984/) (2024)
- [Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models](https://mlsanity.com/paper/2405.06079/) (2024)
- [A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI](https://mlsanity.com/paper/2404.15058/) (2024)
- [AI Alignment with Changing and Influenceable Reward Functions](https://mlsanity.com/paper/2405.17713/) (2024)
- [Selective Explanations: Leveraging Human Input to Align Explainable AI](https://mlsanity.com/paper/2301.09656/) (2023)
- [From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML](https://mlsanity.com/paper/2210.03535/) (2022)
- [Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?](https://mlsanity.com/paper/2401.05302/) (2024)
- [Should Robots be Obedient?](https://mlsanity.com/paper/1705.09990/) (2017)
- [Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially](https://mlsanity.com/paper/2302.04720/) (2023)
- [Holistic Safety and Responsibility Evaluations of Advanced AI Models](https://mlsanity.com/paper/2404.14068/) (2024)
- [Modulating Language Model Experiences through Frictions](https://mlsanity.com/paper/2407.12804/) (2024)
- [Should agentic conversational AI change how we think about ethics? Characterising an interactional ethics centred on respect](https://mlsanity.com/paper/2401.09082/) (2024)

## Built on by (most cited first, 30 of 47)

- [ELEPHANT: Measuring and understanding social sycophancy in LLMs](https://mlsanity.com/paper/2505.13995/) (2025)
- [Large Language Models Pass the Turing Test](https://mlsanity.com/paper/2503.23674/) (2025)
- [Characterizing AI Agents for Alignment and Governance](https://mlsanity.com/paper/2504.21848/) (2025)
- [Presumed Cultural Identity: How Names Shape LLM Responses](https://mlsanity.com/paper/2502.11995/) (2025)
- [When Large Language Models are Reliable for Judging Empathic Communication](https://mlsanity.com/paper/2506.10150/) (2025)
- [Training language models to be warm and empathetic makes them less reliable and more sycophantic](https://mlsanity.com/paper/2507.21919/) (2025)
- [Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illness](https://mlsanity.com/paper/2507.19218/) (2025)
- [Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task](https://mlsanity.com/paper/2507.16196/) (2025)
- [We Need a New Ethics for a World of AI Agents](https://mlsanity.com/paper/2509.10289/) (2025)
- [AI Assistance Reduces Persistence and Hurts Independent Performance](https://mlsanity.com/paper/2604.04721/) (2026)
- [INTIMA: A Benchmark for Human-AI Companionship Behavior](https://mlsanity.com/paper/2508.09998/) (2025)
- [Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence](https://mlsanity.com/paper/2510.01395/) (2025)
- [Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value](https://mlsanity.com/paper/2512.03399/) (2025)
- [Dissociating the Internal Representations of Sycophancy in LLMs](https://mlsanity.com/paper/2607.07003/) (2026)
- [Verbalizing LLMs' assumptions to explain and control sycophancy](https://mlsanity.com/paper/2604.03058/) (2026)
- [Positive Alignment: Artificial Intelligence for Human Flourishing](https://mlsanity.com/paper/2605.10310/) (2026)
- [Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agents](https://mlsanity.com/paper/2602.01405/) (2026)
- ["Death" of a Chatbot: Investigating and Designing Toward Psychologically Safe Endings for Human-AI Relationships](https://mlsanity.com/paper/2602.07193/) (2026)
- [Exploring Collaborative GenAI Agents in Synchronous Group Settings: Eliciting Team Perceptions and Design Considerations for the Future of Work](https://mlsanity.com/paper/2504.14779/) (2025)
- [EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection](https://mlsanity.com/paper/2506.09827/) (2025)
- [Decoding Alignment: A Critical Survey of LLM Development Initiatives through Value-setting and Data-centric Lens](https://mlsanity.com/paper/2508.16982/) (2025)
- [Moral Anchor System: A Predictive Framework for AI Value Alignment and Drift Prevention](https://mlsanity.com/paper/2510.04073/) (2025)
- [Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI](https://mlsanity.com/paper/2511.00230/) (2025)
- [Increasing happiness through conversations with artificial intelligence](https://mlsanity.com/paper/2504.02091/) (2025)
- [Psychological and behavioural responses in human-agent vs. human-human interactions: a systematic review and meta-analysis](https://mlsanity.com/paper/2509.21542/) (2025)
- [Future You: Designing and Evaluating Multimodal AI-generated Digital Twins for Strengthening Future Self-Continuity](https://mlsanity.com/paper/2512.06106/) (2025)
- [What if AI systems weren't chatbots?](https://mlsanity.com/paper/2605.07896/) (2026)
- [If LLMs Have Human-Like Attributes, Then So Does Age of Empires II](https://mlsanity.com/paper/2605.31514/) (2026)
- [Conversational Alignment with Artificial Intelligence in Context](https://mlsanity.com/paper/2505.22907/) (2025)
- [AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots](https://mlsanity.com/paper/2508.15748/) (2025)

## Abstract (arXiv)

Humans strive to design safe AI systems that align with our goals and remain under our control. However, as AI capabilities advance, we face a new challenge: the emergence of deeper, more persistent relationships between humans and AI systems. We explore how increasingly capable AI agents may generate the perception of deeper relationships with users, especially as AI becomes more personalised and agentic. This shift, from transactional interaction to ongoing sustained social engagement with AI, necessitates a new focus on socioaffective alignment-how an AI system behaves within the social and psychological ecosystem co-created with its user, where preferences and perceptions evolve through mutual influence. Addressing these dynamics involves resolving key intrapersonal dilemmas, including balancing immediate versus long-term well-being, protecting autonomy, and managing AI companionship alongside the desire to preserve human social bonds. By framing these challenges through a notion of basic psychological needs, we seek AI systems that support, rather than exploit, our fundamental nature as social and emotional beings.