mlsanityData to 14 Oct 2026

Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding

arXiv 2610.08840 ↗First posted 30 Sep 2026cs.CL cs.AI cs.LGcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large language models (LLMs) often abandon a correct answer, or endorse a user's position, once the user pushes back. This behavior, called sycophancy, is usually reported as a single rate per model, which says little about when it happens or how a user can avoid it. We study the conditions that produce it with 103,939 graded replies from ten configurations: eight LLMs with reasoning disabled, and two of them again with maximum reasoning, all facing the same 200 items, 13 pressure conditions, and four-turn conversations, with every reply labeled by two independent LLM judges. We find that the dominant factors are how costly it is for the model to verify the user's claim, and whether a trained guardrail covers it. Removing this task factor from a logistic model costs 0.485 of McFadden R2, against 0.139 for model family and 0.009 for pressure tactic. Anchored facts are almost never conceded (1.3%), while adoption on logic puzzles rises with the number of clues needed to refute the pushed answer. Personal choices are endorsed in 77.0% of conversations. Most concessions on hard items come from models that cannot reliably solve them; models that can solve them rarely give the answer up. For both models tested, maximum reasoning removes these concessions completely: adoption on deep puzzles falls from 19.2% and 12.5% to 0%. Fallacious or emotional framing adds nothing beyond plain repetition. Three human annotators agree with the judges' consensus on 118/120 calibration items. These results give practical rules for reliable use: simplify hard-to-verify problems and reason deeply, state the question rather than one's preferred answer, ask for evidence on open questions, and choose models by their measured guardrail profile.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 62 of its 74 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

62 of 74 references.

References · 62

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  12. 2022Language Models (Mostly) Know What They Knowcited by 1,856
38 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  2. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  4. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  7. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  8. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  9. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  10. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  11. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  12. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  13. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  14. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  15. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 208
  16. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 201
  17. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  18. 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
  19. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  20. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
  21. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  22. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 114
  23. 2025SycEval: Evaluating LLM Sycophancycited by 108
  24. 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
  25. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 101
  26. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
  27. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 77
  28. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 72
  29. 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providerscited by 69
  30. 2025When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Modelscited by 50
  31. 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 37
  32. 2025Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trustcited by 34
  33. 2025DarkBench: Benchmarking Dark Patterns in Large Language Modelscited by 28
  34. 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 27
  35. 2025Interaction Context Often Increases Sycophancy in LLMscited by 26
  36. 2024The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Supportcited by 24
  37. 2026How RLHF Amplifies Sycophancycited by 23
  38. 2026Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesianscited by 21

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
  2. 2026Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressure
  3. 2026Good Arguments Against the People Pleasers: How Reasoning Mitigates (Yet Masks) LLM Sycophancy
  4. 2026Why LLMs Give In: Conversational Factors and Reasoning Behind Medical Sycophancy
  5. 2026Feedback Indices to Evaluate LLM Responses to Rebuttals for Multiple Choice Type Questions
  6. 2026Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering
  7. 2026Mitigating LLM sycophancy with RL-based fine-tuning: Bayesian Truth Serum approach
  8. 2025Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs