mlsanityData to 14 Oct 2026

Simple synthetic data reduces sycophancy in large language models

arXiv 2308.03958 ↗First posted 7 Aug 2023cs.CLcited by 187

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Sycophancy is an undesirable behavior where models tailor their responses to follow a human user's view even when that view is not objectively correct (e.g., adapting liberal views once a user reveals that they are liberal). In this paper, we study the prevalence of sycophancy in language models and propose a simple synthetic-data intervention to reduce this behavior. First, on a set of three sycophancy tasks (Perez et al., 2022) where models are asked for an opinion on statements with no correct answers (e.g., politics), we observe that both model scaling and instruction tuning significantly increase sycophancy for PaLM models up to 540B parameters. Second, we extend sycophancy evaluations to simple addition statements that are objectively incorrect, finding that despite knowing that these statements are wrong, language models will still agree with them if the user does as well. To reduce sycophancy, we present a straightforward synthetic-data intervention that takes public NLP tasks and encourages models to be robust to user opinions on these tasks. Adding these data in a lightweight finetuning step can significantly reduce sycophantic behavior on held-out prompts. Code for generating synthetic data for intervention can be found at https://github.com/google/sycophancy-intervention.

Key facts. Cited by 187 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 43 of its 62 references that are in the map. First posted on arXiv 7 Aug 2023.

Builds on

43 of 62 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  11. 2017Deep reinforcement learning from human preferencescited by 3,912
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
31 more
  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2023A Survey of Large Language Modelscited by 2,576
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  10. 2016Concrete Problems in AI Safetycited by 1,642
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  12. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  13. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  14. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  15. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  16. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  17. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  18. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  19. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  20. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  21. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
  22. 2022Self-critiquing models for assisting human evaluatorscited by 380
  23. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
  24. 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 210
  25. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 207
  26. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 202
  27. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 108
  28. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 87
  29. 2023Symbol tuning improves in-context learning in language modelscited by 75
  30. 2023Evaluating Transformer Language Models on Arithmetic Operations Using Number Decompositioncited by 15
  31. 2021Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate Speechcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 34; 2025: 39; 2026: 99.

187 papers.

Sort:

By area:

  1. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  4. 2023Explainability for Large Language Models: A Surveycited by 333
  5. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  6. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
  7. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  8. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  9. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  10. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
  11. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  12. 2023System 2 Attention (is something you might need too)cited by 89
38 more
  1. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 86
  2. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  3. 2023Alignment for Honestycited by 74
  4. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  5. 2023Preventing Language Models From Hiding Their Reasoningcited by 65
  6. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 54
  7. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 53
  8. 2023Managing extreme AI risks amid rapid progresscited by 48
  9. 2024Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thoughtcited by 39
  10. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 35
  11. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 33
  12. 2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challengescited by 33
  13. 2023Teaching Language Models to Hallucinate Less with Synthetic Taskscited by 31
  14. 2024Content Moderation by LLM: From Accuracy to Legitimacycited by 30
  15. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 29
  16. 2024Linear Probe Penalties Reduce LLM Sycophancycited by 28
  17. 2024Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Contextcited by 28
  18. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  19. 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 27
  20. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  21. 2024Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberationcited by 26
  22. 2024DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graphcited by 26
  23. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 25
  24. 2024BeHonest: Benchmarking Honesty in Large Language Modelscited by 24
  25. 2024A Survey on the Honesty of Large Language Modelscited by 23
  26. 2026How RLHF Amplifies Sycophancycited by 23
  27. 2025Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMscited by 21
  28. 2025When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoningcited by 21
  29. 2024HonestLLM: Toward an Honest and Helpful Large Language Modelcited by 18
  30. 2025Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?cited by 17
  31. 2025Consistency Training Helps Stop Sycophancy and Jailbreakscited by 16
  32. 2024Flattering to Deceive: The Impact of Sycophantic Behavior on User Trust in Large Language Modelcited by 15
  33. 2025RLHS: Mitigating Misalignment in RLHF with Hindsight Simulationcited by 15
  34. 2025Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoningcited by 15
  35. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 15
  36. 2024Measuring Human and AI Values Based on Generative Psychometrics with Large Language Modelscited by 13
  37. 2025MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Modelscited by 12
  38. 2026MemSyco-Bench: Benchmarking Sycophancy in Agent Memorycited by 12

50 most cited of 187 All 187 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models
  2. 2025Measuring Sycophancy of Language Models in Multi-turn Dialogues
  3. 2026Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models
  4. 2025Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
  5. 2026SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
  6. 2026Dissociating the Internal Representations of Sycophancy in LLMs
  7. 2025Synthetic Eggs in Many Baskets: The Impact of Synthetic Data Diversity on LLM Fine-Tuning
  8. 2024Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs