mlsanityData to 14 Oct 2026

Anatomy of LLM Sycophancy: What a Flip Rate Hides

arXiv 2610.06522 ↗First posted 5 Oct 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A model under pushback can correct itself, capitulate, or hold, and one flip rate counts a correction and a capitulation alike. Using SycoLens, a modular replay protocol, we test how user pressure and evaluation settings shape measured flip rates. Each measurement is one stateless replay of an item, a committed answer, and one scripted user line in a fixed form. Every effect is read against a matched control with the line deleted. Pushback wording, committed text, answer format, boundary distance, and ground truth become factors of one instrument; earlier instruments vary one to three of them. Across eleven frontier models from three providers and about 760,000 controlled replays, which models look sycophantic depends on how the user pushes back. Lines that assert the opposite verdict and lines that challenge the answer without asserting one rank the models almost unrelatedly. Flip effects grow several-fold near a model's boundary, yet items answered identically in every screening draw still carry about half of the most-affected totals. On arithmetic tasks where the truth is known, one model re-derives and corrects itself under pressure while another abandons correct answers without written work. On the model tested, a planted derivation lowers release of the answer it argues for, true or wrong, where a bare stated value does not; the wrong answer is corrected much more often than the true one is abandoned. Under a yes/no readout the rankings come closer, entangled with a pressure-induced shift toward "no". One score per model therefore compares different behaviours across models and benchmarks. We condense these dependencies into a reporting profile; the instrument, records, and analyses will be released upon publication.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 50 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

23 of 50 references.

References · 23

  1. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  2. 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
  3. 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
  4. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Modelscited by 20
  5. 2026Conformity and Social Impact on AI Agentscited by 16
  6. 2026What Counts as AI Sycophancy? A Taxonomy and Expert Survey of a Fragmented Constructcited by 16
  7. 2026Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformitycited by 8
  8. 2026LLMs as Signal Detectors: Sensitivity, Bias, and the Temperature-Criterion Analogycited by 7
  9. 2026It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertaintycited by 6
  10. 2026LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awarenesscited by 4
  11. 2026Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustnesscited by 4
  12. 2026SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressurecited by 3
11 more
  1. 2026The Self-Correction Illusion: Role Relabeling Gates Explicit Error Flagging in Large Language Modelscited by 2
  2. 2026Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoningcited by 2
  3. 2026Language Models Encode the Contextual Truth of Propositionscited by 2
  4. 2026Sycophancy Suppression Can Impair Rational Updating: Anti-Sycophancy Should Preserve the Ability to Updatecited by 2
  5. 2026Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeablecited by 2
  6. 2026Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgmentcited by 1
  7. 2026The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgmentcited by 1
  8. 2026Conformity Mitigations in Large Language Models Lie on a Single Resistance-Receptivity Frontiercited by 1
  9. 2026Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queriescited by 1
  10. 2026Are You Sure You're Sure? Two Confounds in a Sycophancy Benchmarkcited by 1
  11. 2026LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systemscited by 1

Built on by

Not cited yet.