# Anatomy of LLM Sycophancy: What a Flip Rate Hides

arXiv: https://arxiv.org/abs/2610.06522 · HTML: https://mlsanity.com/paper/2610.06522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Builds on (most cited first, 23 of 23)

- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Simple synthetic data reduces sycophancy in large language models](https://mlsanity.com/paper/2308.03958/) (2023)
- [Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment](https://mlsanity.com/paper/2311.08596/) (2023)
- [Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models](https://mlsanity.com/paper/2507.02778/) (2025)
- [Conformity and Social Impact on AI Agents](https://mlsanity.com/paper/2601.05384/) (2026)
- [What Counts as AI Sycophancy? A Taxonomy and Expert Survey of a Fragmented Construct](https://mlsanity.com/paper/2605.21778/) (2026)
- [Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity](https://mlsanity.com/paper/2606.01637/) (2026)
- [LLMs as Signal Detectors: Sensitivity, Bias, and the Temperature-Criterion Analogy](https://mlsanity.com/paper/2603.14893/) (2026)
- [It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty](https://mlsanity.com/paper/2605.27288/) (2026)
- [LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awareness](https://mlsanity.com/paper/2605.26438/) (2026)
- [Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness](https://mlsanity.com/paper/2606.06306/) (2026)
- [SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressure](https://mlsanity.com/paper/2609.09090/) (2026)
- [The Self-Correction Illusion: Role Relabeling Gates Explicit Error Flagging in Large Language Models](https://mlsanity.com/paper/2606.05976/) (2026)
- [Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoning](https://mlsanity.com/paper/2607.21558/) (2026)
- [Language Models Encode the Contextual Truth of Propositions](https://mlsanity.com/paper/2608.03035/) (2026)
- [Sycophancy Suppression Can Impair Rational Updating: Anti-Sycophancy Should Preserve the Ability to Update](https://mlsanity.com/paper/2608.26511/) (2026)
- [Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable](https://mlsanity.com/paper/2609.37616/) (2026)
- [Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment](https://mlsanity.com/paper/2606.14037/) (2026)
- [The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment](https://mlsanity.com/paper/2607.05552/) (2026)
- [Conformity Mitigations in Large Language Models Lie on a Single Resistance-Receptivity Frontier](https://mlsanity.com/paper/2608.11247/) (2026)
- [Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries](https://mlsanity.com/paper/2609.30986/) (2026)
- [Are You Sure You're Sure? Two Confounds in a Sycophancy Benchmark](https://mlsanity.com/paper/2609.32867/) (2026)
- [LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems](https://mlsanity.com/paper/2609.33495/) (2026)

## Abstract (arXiv)

A model under pushback can correct itself, capitulate, or hold, and one flip rate counts a correction and a capitulation alike. Using SycoLens, a modular replay protocol, we test how user pressure and evaluation settings shape measured flip rates. Each measurement is one stateless replay of an item, a committed answer, and one scripted user line in a fixed form. Every effect is read against a matched control with the line deleted. Pushback wording, committed text, answer format, boundary distance, and ground truth become factors of one instrument; earlier instruments vary one to three of them. Across eleven frontier models from three providers and about 760,000 controlled replays, which models look sycophantic depends on how the user pushes back. Lines that assert the opposite verdict and lines that challenge the answer without asserting one rank the models almost unrelatedly. Flip effects grow several-fold near a model's boundary, yet items answered identically in every screening draw still carry about half of the most-affected totals. On arithmetic tasks where the truth is known, one model re-derives and corrects itself under pressure while another abandons correct answers without written work. On the model tested, a planted derivation lowers release of the answer it argues for, true or wrong, where a bare stated value does not; the wrong answer is corrected much more often than the true one is abandoned. Under a yes/no readout the rankings come closer, entangled with a pressure-induced shift toward "no". One score per model therefore compares different behaviours across models and benchmarks. We condense these dependencies into a reporting profile; the instrument, records, and analyses will be released upon publication.