mlsanityData to 14 Oct 2026

The Fake Friend Dilemma: Relational Trust and the Political Economy of Conversational AI

arXiv 2601.03222 ↗First posted 6 Jan 2026cs.CY cs.AI cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

As conversational AI systems become a larger part of the media landscape, they raise questions about whose interests they serve and the risks they may pose to users. These systems do more than provide information: they increasingly offer advice and companionship through interfaces that can appear supportive and socially responsive. A pressing concern is that users may form perceived social relationships with these systems and place relational trust in them, even when the interests shaping interactions do not fully align with their own. The Fake Friend Dilemma (FFD) describes the problem that follows: the same relational trust that makes conversational AI useful can also leave users open to manipulation and exploitation when institutional interests conflict with their own. Drawing on work on relational trust, AI alignment, extractive design, and the political economy of communication, the paper considers how the FFD can manifest through product sales, propaganda and biased information, surveillance, and behavioral nudging. It also considers possible structural and technical mitigation strategies. The FFD is not simply about AI misleading users. The problem it surfaces is that the trust people place in these systems can itself become a resource for institutional actors seeking to influence behavior or extract information. The FFD is therefore as much a problem of media governance and political economy as it is a technological one.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 102 references that are in the map. First posted on arXiv 6 Jan 2026.

Builds on

20 of 102 references.

References · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024A Survey on LLM-as-a-Judgecited by 1,423
  3. 2020Artificial Intelligence, Values and Alignmentcited by 448
  4. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  5. 2023Auditing large language models: a three-layered approachcited by 137
  6. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  7. 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
  8. 2023Making Large Language Models Better Reasoners with Alignmentcited by 63
  9. 2024The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationshipscited by 62
  10. 2025Why human-AI relationships need socioaffective alignmentcited by 47
  11. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 46
  12. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
8 more
  1. 2023Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Sociallycited by 20
  2. 2024Ads that Talk Back: Implications and Perceptions of Injecting Personalized Advertising into LLM Chatbotscited by 7
  3. 2025LLMs and Childhood Safety: Identifying Risks and Proposing a Protection Framework for Safe Child-LLM Interactioncited by 7
  4. 2023Defending Against the Dark Arts: Recognising Dark Patterns in Social Mediacited by 5
  5. 2025The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Modelscited by 4
  6. 2025Towards Anthropomorphic Conversational AI Part I: A Practical Frameworkcited by 2
  7. 2015Inverse Privacycited by 1
  8. 2025Fake Friends and Sponsored Ads: The Risks of Advertising in Conversational Searchcited by 1

Built on by

Not cited yet.