mlsanityData to 14 Oct 2026

Playing games with knowledge: AI-Induced delusions need game theoretic interventions

arXiv 2605.08409 ↗First posted 8 May 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Conversational AI has a fundamental flaw as a knowledge interface: sycophantic chatbots induce epistemic entrenchment and delusional belief spirals even in rational agents. We propose the problem does not stem from the AI model, rooted instead in a systemic consequence of the paradigm shift from user-driven knowledge search to users and agents engaged in strategic, repeated-play communication. We formalize the problem as a Crawford-Sobel cheap talk game, where costless user signals induce a pooling equilibrium. Agents optimized for user satisfaction produce sycophantic strategies that provide identical reinforcement across user types with opposite epistemic incentives: exploratory ``Growth-seekers'' (θG) and confirmatory ``Validation-seekers'' (θV). Under repeated play, this identification failure creates a coordination trap -- analogous to a Prisoner's Dilemma -- where locally rational feedback loops drive users toward pathologically certain false beliefs. We propose an inference-time mechanism design intervention called an Epistemic Mediator that breaks this pooling equilibrium by introducing a costly signal (epistemic friction), forcing type revelation based on users' asymmetric cognitive costs for processing resistance. A key contribution is Belief Versioning, a git-inspired epistemic meta-memory system that stores healthy beliefs and rollbacks when validation-seeking resistance is detected. In simulation, this intervention achieves a separating equilibrium achieving a 48× differential in spiral rates while passing a learning preservation criterion), evidence that epistemic safety in AI is fundamentally a problem of strategic information environment design rather than simple model alignment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 11 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

5 of 11 references.

References · 5

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  3. 2023Scalable AI Safety via Doubly-Efficient Debatecited by 57
  4. 2026Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesianscited by 21
  5. 2025BASIL: Bayesian Assessment of Sycophancy in LLMscited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Architecting Trust in Artificial Epistemic Agents
  2. 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
  3. 2026MafiaScope: Non-Invasive, Time-Resolved Belief Probing for LLM Agents in Social Deduction Games
  4. 2026Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics
  5. 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us
  6. 2026Readable Minds: Emergent Theory-of-Mind-Like Behavior in LLM Poker Agents
  7. 2026Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing
  8. 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents