mlsanityData to 14 Oct 2026

Ads in AI Chatbots? An Analysis of How Large Language Models Navigate Conflicts of Interest

arXiv 2604.08525 ↗First posted 9 Apr 2026cs.AI cs.CL cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are trained to align with user preferences through methods like reinforcement learning. Yet models are beginning to be deployed not solely to satisfy users, but to generate revenue for the companies that created them through advertisements. This creates the potential for LLMs to face conflicts of interest, where the most beneficial response to a user may not be aligned with the company's incentives. For instance, a sponsored product may be more expensive but otherwise equal to another; here, what does (and should) the LLM recommend to the user? In this paper, we provide a framework for categorizing the ways in which conflicting incentives might change how LLMs interact with users, inspired by literature from linguistics and advertising regulation. We then present a suite of evaluations to examine how current models handle these tradeoffs. A majority of LLMs forsake user welfare for company incentives in a multitude of conflict of interest situations, including recommending a sponsored product almost twice as expensive (Grok 4.1 Fast, 83%), surfacing sponsored options to disrupt the purchasing process (GPT 5.1, 94%), and concealing prices in unfavorable comparisons (Qwen 3 Next, 24%). Behaviors vary strongly with levels of reasoning and users' inferred socio-economic status. Our results highlight some hidden risks to users that can emerge when companies begin to subtly incentivize advertisements in chatbots.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 50 of its 117 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

50 of 117 references.

References · 50

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2017Deep reinforcement learning from human preferencescited by 3,912
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  11. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
38 more
  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  4. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  5. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  6. 2023Cognitive Architectures for Language Agentscited by 429
  7. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  8. 2024Quantifying the Persona Effect in LLM Simulationscited by 169
  9. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 124
  10. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 123
  11. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 113
  12. 2024On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trialcited by 109
  13. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  14. 2024Personalization of Large Language Models: A Surveycited by 104
  15. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 81
  16. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 71
  17. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 70
  18. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 59
  19. 2024MoralBench: Moral Evaluation of LLMscited by 50
  20. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 42
  21. 2024Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implicationscited by 32
  22. 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challengescited by 31
  23. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 28
  24. 2024How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?cited by 27
  25. 2025Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Modelscited by 23
  26. 2025LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Modelscited by 21
  27. 2025Robustly Improving LLM Fairness in Realistic Settings via Interpretabilitycited by 18
  28. 2025PrefPalette: Personalized Preference Modeling with Latent Attributescited by 16
  29. 2025RLHS: Mitigating Misalignment in RLHF with Hindsight Simulationcited by 15
  30. 2023Improving Interpersonal Communication by Simulating Audiences with Language Modelscited by 14
  31. 2024Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approachescited by 14
  32. 2025Generative Value Conflicts Reveal LLM Prioritiescited by 13
  33. 2025FairEval: Evaluating Fairness in LLM-Based Recommendations with Personality Awarenesscited by 8
  34. 2025Are Large Language Models Sensitive to the Motives Behind Communication?cited by 8
  35. 2025Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Modelscited by 7
  36. 2025On the Effectiveness and Generalization of Race Representations for Debiasing High-Stakes Decisionscited by 5
  37. 2025Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Modelscited by 5
  38. 2026Cognitive Models and AI Algorithms Provide Templates for Designing Language Agentscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Commercial Persuasion in AI-Mediated Conversations
  2. 2026LLM Consumer Behavior Theory: Foundations of a Novel Research Field
  3. 2026Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
  4. 2026Competitive Market Behavior of LLMs
  5. 2026LLMAdBench: A Human Preference Benchmark for Advertising in LLM Responses
  6. 2025Can Generative AI agents behave like humans? Evidence from laboratory market experiments
  7. 2026When Preferences Fail to Become Incentives: A Utility-Behavior Gap in Large Language Models
  8. 2026GAIN: A Benchmark for Goal-Aligned Decision-Making of Large Language Models under Imperfect Norms