# Accounting for Sycophancy in Language Model Uncertainty Estimation

arXiv: https://arxiv.org/abs/2410.14746 · HTML: https://mlsanity.com/paper/2410.14746/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Unsupervised Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2005.10608/) (2020)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Out-of-Distribution Detection and Selective Generation for Conditional Language Models](https://mlsanity.com/paper/2209.15558/) (2022)
- [LM-Polygraph: Uncertainty Estimation for Language Models](https://mlsanity.com/paper/2311.07383/) (2023)
- [Conversations Gone Awry: Detecting Early Signs of Conversational Failure](https://mlsanity.com/paper/1805.05345/) (2018)
- [CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems](https://mlsanity.com/paper/2103.15721/) (2021)
- [Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop](https://mlsanity.com/paper/1909.01362/) (2019)
- [Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes](https://mlsanity.com/paper/1907.00326/) (2019)
- [Dynamic Forecasting of Conversation Derailment](https://mlsanity.com/paper/2110.05111/) (2021)
- [PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners](https://mlsanity.com/paper/2207.05685/) (2022)
- [The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error](https://mlsanity.com/paper/2203.11317/) (2022)
- [Learning to Generate Equitable Text in Dialogue from Biased Training Data](https://mlsanity.com/paper/2307.04303/) (2023)

## Built on by (most cited first, 18 of 18)

- [When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models](https://mlsanity.com/paper/2508.02087/) (2025)
- [What Counts as AI Sycophancy? A Taxonomy and Expert Survey of a Fragmented Construct](https://mlsanity.com/paper/2605.21778/) (2026)
- [Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illness](https://mlsanity.com/paper/2507.19218/) (2025)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [BASIL: Bayesian Assessment of Sycophancy in LLMs](https://mlsanity.com/paper/2508.16846/) (2025)
- [How Overconfidence in Initial Choices and Underconfidence Under Criticism Modulate Change of Mind in Large Language Models](https://mlsanity.com/paper/2507.03120/) (2025)
- [PARROT: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMs](https://mlsanity.com/paper/2511.17220/) (2025)
- [SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy](https://mlsanity.com/paper/2604.02423/) (2026)
- [Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation](https://mlsanity.com/paper/2510.02279/) (2025)
- [It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty](https://mlsanity.com/paper/2605.27288/) (2026)
- [How to Mitigate Overfitting in Weak-to-strong Generalization?](https://mlsanity.com/paper/2503.04249/) (2025)
- [Uncertainty Quantification for Evaluating Machine Translation Bias](https://mlsanity.com/paper/2507.18338/) (2025)
- [Sycophancy Suppression Can Impair Rational Updating: Anti-Sycophancy Should Preserve the Ability to Update](https://mlsanity.com/paper/2608.26511/) (2026)
- [Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation](https://mlsanity.com/paper/2510.09390/) (2025)
- [Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering](https://mlsanity.com/paper/2607.28906/) (2026)
- [Don't `Well, Actually' Me Unless You Know What You're Talking About: Weak Presupposition Verification Degrades General QA Performance](https://mlsanity.com/paper/2608.06539/) (2026)
- [Conversational Control with Ontologies for Large Language Models: A Lightweight Framework for Constrained Generation](https://mlsanity.com/paper/2604.04450/) (2026)
- [Pander Score: A Continuous Measure of Sycophancy as Epistemic Deference](https://mlsanity.com/paper/2606.07897/) (2026)

## Abstract (arXiv)

Effective human-machine collaboration requires machine learning models to externalize uncertainty, so users can reflect and intervene when necessary. For language models, these representations of uncertainty may be impacted by sycophancy bias: proclivity to agree with users, even if they are wrong. For instance, models may be over-confident in (incorrect) problem solutions suggested by a user. We study the relationship between sycophancy and uncertainty estimation for the first time. We propose a generalization of the definition of sycophancy bias to measure downstream impacts on uncertainty estimation, and also propose a new algorithm (SyRoUP) to account for sycophancy in the uncertainty estimation process. Unlike previous works on sycophancy, we study a broad array of user behaviors, varying both correctness and confidence of user suggestions to see how model answers (and their certainty) change. Our experiments across conversation forecasting and question-answering tasks show that user confidence plays a critical role in modulating the effects of sycophancy, and that SyRoUP can better predict these effects. From these results, we argue that externalizing both model and user uncertainty can help to mitigate the impacts of sycophancy bias.