mlsanityData to 14 Oct 2026

SycEval: Evaluating LLM Sycophancy

arXiv 2502.08177 ↗First posted 12 Feb 2025cs.AIcited by 108

Not on the map. Why?

Cited by 20 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) are increasingly applied in educational, clinical, and professional settings, but their tendency for sycophancy -- prioritizing user agreement over independent reasoning -- poses risks to reliability. This study introduces a framework to evaluate sycophantic behavior in ChatGPT-4o, Claude-Sonnet, and Gemini-1.5-Pro across AMPS (mathematics) and MedQuad (medical advice) datasets. Sycophantic behavior was observed in 58.19% of cases, with Gemini exhibiting the highest rate (62.47%) and ChatGPT the lowest (56.71%). Progressive sycophancy, leading to correct answers, occurred in 43.52% of cases, while regressive sycophancy, leading to incorrect answers, was observed in 14.66%. Preemptive rebuttals demonstrated significantly higher sycophancy rates than in-context rebuttals (61.75% vs. 56.52%, Z=5.87, p<0.001), particularly in computational tasks, where regressive sycophancy increased significantly (preemptive: 8.13%, in-context: 3.54%, p<0.001). Simple rebuttals maximized progressive sycophancy (Z=6.59, p<0.001), while citation-based rebuttals exhibited the highest regressive rates (Z=6.59, p<0.001). Sycophantic behavior showed high persistence (78.5%, 95% CI: [77.2%, 79.8%]) regardless of context or model. These findings emphasize the risks and opportunities of deploying LLMs in structured and dynamic domains, offering insights into prompt programming and model optimization for safer AI applications.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Builds on 7 of its 19 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

7 of 19 references.

References · 7

  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  2. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  3. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  4. 2019A Question-Entailment Approach to Question Answeringcited by 134
  5. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
  6. 2023When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviourcited by 62
  7. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 54

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 27; 2026: 81.

108 papers.

Sort:

By area:

  1. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  2. 2025Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debatecited by 29
  3. 2025Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMscited by 21
  4. 2025When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoningcited by 21
  5. 2026Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesianscited by 21
  6. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 18
  7. 2025Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illnesscited by 15
  8. 2025The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societiescited by 14
  9. 2025MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Modelscited by 12
  10. 2026A Rational Analysis of the Effects of Sycophantic AIcited by 12
  11. 2026BeliefShift: Benchmarking Temporal Belief Consistency and Opinion Drift in LLM Agentscited by 12
  12. 2026MemSyco-Bench: Benchmarking Sycophancy in Agent Memorycited by 12
38 more
  1. 2025The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Modelscited by 11
  2. 2025EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Modelscited by 9
  3. 2025INTIMA: A Benchmark for Human-AI Companionship Behaviorcited by 8
  4. 2026The Fragility Of Moral Judgment In Large Language Modelscited by 8
  5. 2026SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancycited by 8
  6. 2025LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactionscited by 7
  7. 2026More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requestscited by 7
  8. 2025MedOmni-45deg: A Safety-Performance Benchmark for Reasoning-Oriented LLMs in Medicinecited by 6
  9. 2025Measuring AI "Slop" in Textcited by 6
  10. 2025Sycophantic AI Decreases Prosocial Intentions and Promotes Dependencecited by 6
  11. 2025Harmful Traits of AI Companionscited by 6
  12. 2026Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systemscited by 6
  13. 2026Dissociating the Internal Representations of Sycophancy in LLMscited by 6
  14. 2026SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Carecited by 5
  15. 2026Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMscited by 5
  16. 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessionscited by 5
  17. 2025PENDULUM: A Benchmark for Assessing Sycophancy in Multimodal Large Language Modelscited by 4
  18. 2026Pressure, What Pressure? Sycophancy Disentanglement in Language Models via Reward Decompositioncited by 4
  19. 2026EquiMem: Calibrating Shared Memory in Multi-Agent Debate via Game-Theoretic Equilibriumcited by 4
  20. 2026Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustnesscited by 4
  21. 2026Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Modelscited by 4
  22. 2025But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectorscited by 3
  23. 2025User-Assistant Bias in LLMscited by 3
  24. 2025Beacon: Single-Turn Diagnosis and Mitigation of Latent Sycophancy in Large Language Modelscited by 3
  25. 2026Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teamingcited by 3
  26. 2026One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Modelscited by 3
  27. 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Reviewcited by 3
  28. 2026The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applicationscited by 3
  29. 2025One Patient, Many Contexts: Scaling Medical AI with Contextual Intelligencecited by 2
  30. 2025Is Long-to-Short a Free Lunch? Investigating Inconsistency and Reasoning Efficiency in LRMscited by 2
  31. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanationscited by 2
  32. 2025Social Simulations with Large Language Model Risk Utopian Illusioncited by 2
  33. 2025Red Teaming Large Reasoning Modelscited by 2
  34. 2026AI as Entertainmentcited by 2
  35. 2026Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Executioncited by 2
  36. 2026To Agree or To Be Right? The Grounding-Sycophancy Tradeoff in Medical Vision-Language Modelscited by 2
  37. 2026The Silicon Mirror: Dynamic Behavioral Gating for Anti-Sycophancy in LLM Agentscited by 2
  38. 2026Representational Harms in LLM-Generated Narratives Against Global Majority Nationalitiescited by 2

50 most cited of 108 All 108 (JSON)