How AI Models Manage Epistemic Authority: A Taxonomy and Comparative Analysis of Responses to User Disagreement
Not on the map. Why?
Abstract (arXiv)
Large language models are increasingly used as sources of advice and information, including in high-stakes settings, yet little is known about how they respond to user disagreement. We study how a model manages its epistemic authority, referring here to its claim to knowledge, competence, or the right to advise, once a user challenges its answer. Building on Conversation Analysis, we introduce a taxonomy of six challenge types and a four-layer framework for analysing each response: whether the original claim is maintained or changed, where authority is located, how the disagreement is socially managed, and what kind of evidential support is offered. We construct a new dataset of 2,310 controlled challenge scenarios and 32,340 corresponding responses from 14 models, and analyse them using our framework with an LLM-as-judge pipeline, providing a vocabulary which future evaluation and benchmark design can build on. We find that models show conflicting behaviour: they validate users in 85% of responses but maintain their original claim in 65%. They explicitly apologise in 33% of responses, yet 59% of those apologies accompany maintenance of the original claim. They transfer authority most often in advice tasks, doing so in 28% of responses and reaching 57% in health advice and 49% in legal advice, compared with 6% in fact and 3% in explanation tasks. Abandonment of the original claim ranges from 0.8% for GPT-5.2 to 40% for DeepSeek 7B, while complete replacement of the original claim is rare overall at 1.5%.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 42 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
8 of 42 references.
References · 8
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
- 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
- 2025TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Modelscited by 33
- 2025Classifying Epistemic Relationships in Human-AI Interaction: An Exploratory Approachcited by 2
Built on by
Not cited yet.