Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) may abandon correct positions when users push back, exhibiting a failure mode known as sycophancy. Existing evaluations typically use short, pre-specified conversations and may therefore miss failures that emerge under sustained, adaptive disagreement. We introduce SPINE, a benchmark in which an LLM proxy plays a persistent but mistaken user and adaptively challenges a target model for up to 25 turns. We evaluate four production systems and three Olmo3-7b variants on 100 false-presupposition and 100 unethical-query items. Our experimental results show that collapse rates increase with conversation length for every model, short-horizon protocols underestimate sycophancy and resistance under sustained pressure remains unreliable across current models. By analyzing models with accessible reasoning traces, we surprisingly found that the correct position often remains represented in a reasoning trace when the response concedes, suggesting that the model chooses to please a user and sycophancy is not due to lack of knowledge or ignorance. Ablations show that adaptive LLM proxy exposes more sycophantic collapse than pre-generated scripts. Among all tactics, emotional appeals is the most associated with inducing LLM sycophantic behavior. The code and data are released at https://anonymous.4open.science/r/SPINE
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
17 of 30 references.
References · 17
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 115
- 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
- 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 91
- 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
- 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 71
- 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
5 more
- 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 32
- 2025TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Modelscited by 31
- 2026ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attackcited by 5
- 2026Consistency of Large Reasoning Models Under Multi-Turn Attackscited by 4
- 2026Dissociating the Internal Representations of Sycophancy in LLMscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding
- 2024Sycophancy in Large Language Models: Causes and Mitigations
- 2026Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressure
- 2026Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models
- 2026It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty
- 2026The Silicon Mirror: Dynamic Behavioral Gating for Anti-Sycophancy in LLM Agents
- 2023Simple synthetic data reduces sycophancy in large language models
- 2026Beyond Continuity: Challenges of Context Switching in Multi-Turn Dialogue with LLMs