mlsanityData to 14 Oct 2026

Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy

arXiv 2605.12991 ↗First posted 13 May 2026cs.LG cs.AIcited by 3

On the map: LLM agents and tool use → Social simulation and multi-agent debate

also in 1 more
Abstract (arXiv)

LLM-based multi-agent pipelines flip from correct to incorrect answers under simulated peer disagreement at rates we term yield, a vulnerability widely attributed to RLHF-induced sycophancy. We test this attribution across four model families and find it largely wrong: pretrained base models exhibit the same substitution pattern as their Instruct variants, averaging higher yield than Instruct. Using activation patching, we localize the corruption to a narrow mid-layer window where attention carries the causal weight and MLP contribution is negligible; patching above this window restores 96% of the clean-to-pressured P(correct) gap. The attack surface decomposes into two independent factors (channel framing and consensus strength) whose interaction produces a 47.5 percentage-point yield gap at majority consensus, preserved across jury sizes N ∈ 4, 5, 6. Two converging activation-space interventions show that pressure suppresses clean-reasoning features rather than activating a new sycophancy circuit. A single correctly-arguing dissenter reduces yield by 54-73 percentage points across all framings tested, whereas the strongest prompt-level defense fails on attack variants outside its design surface. Mitigations should target the mechanism, structured dissent at the pipeline level, rather than prompt-level defenses.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 53 of its 63 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

53 of 63 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration18 Apr
  2. 2025When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoning8 Oct
  3. 2025Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debate27 Sep
  4. 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate5 Sep

Other references · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  11. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
37 more
  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  7. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  8. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  9. 2024Yi: Open Foundation Models by 01.AIcited by 719
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  12. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  13. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  14. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  15. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  16. 2018AI safety via debatecited by 400
  17. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  18. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273
  19. 2025Small Language Models are the Future of Agentic AIcited by 270
  20. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
  21. 2024How to use and interpret activation patchingcited by 214
  22. 2024More Agents Is All You Needcited by 180
  23. 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
  24. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
  25. 2025When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Modelscited by 50
  26. 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 37
  27. 2024Conformity in Large Language Modelscited by 34
  28. 2026Sycophancy Hides Linearly in the Attention Headscited by 23
  29. 2026How RLHF Amplifies Sycophancycited by 23
  30. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 19
  31. 2026Ask don't tell: Reducing sycophancy in large language modelscited by 14
  32. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 14
  33. 2025How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidencecited by 12
  34. 2024Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Modelscited by 11
  35. 2025A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesiscited by 9
  36. 2025Automating Deception: Scalable Multi-Turn LLM Jailbreakscited by 7
  37. 2026DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deferencecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate
  2. 2026Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems
  3. 2026Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models
  4. 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
  5. 2026LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuit
  6. 2025Internal Reasoning vs. External Control: A Thermodynamic Analysis of Sycophancy in Large Language Models
  7. 2026Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energy
  8. 2026Beyond Consensus: Trace-Level Synthesis in Mixture of Agents