# Conformity in Large Language Models

arXiv: https://arxiv.org/abs/2410.12428 · HTML: https://mlsanity.com/paper/2410.12428/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 65 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [LUQ: Long-text Uncertainty Quantification for LLMs](https://mlsanity.com/paper/2403.20279/) (2024)
- [Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration](https://mlsanity.com/paper/2405.14314/) (2024)
- [Large Language Model-based Human-Agent Collaboration for Complex Task Solving](https://mlsanity.com/paper/2402.12914/) (2024)
- [A Trip Towards Fairness: Bias and De-Biasing in Large Language Models](https://mlsanity.com/paper/2305.13862/) (2023)
- [Social Influence and the Collective Dynamics of Opinion Formation](https://mlsanity.com/paper/1311.3475/) (2013)
- [Do We Need Language-Specific Fact-Checking Models? The Case of Chinese](https://mlsanity.com/paper/2401.15498/) (2024)
- [See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses](https://mlsanity.com/paper/2408.08978/) (2024)

## Built on by (most cited first, 30 of 34)

- [Free-MAD: Consensus-Free Multi-Agent Debate](https://mlsanity.com/paper/2509.11035/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2508.05687/) (2025)
- [LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions](https://mlsanity.com/paper/2508.18321/) (2025)
- [Conformity and Social Impact on AI Agents](https://mlsanity.com/paper/2601.05384/) (2026)
- [Confidence Estimation for LLMs in Multi-turn Interactions](https://mlsanity.com/paper/2601.02179/) (2026)
- [Are Large Language Models Sensitive to the Motives Behind Communication?](https://mlsanity.com/paper/2510.19687/) (2025)
- [A Review of Incorporating Psychological Theories in LLMs](https://mlsanity.com/paper/2505.00003/) (2025)
- [Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2601.21742/) (2026)
- [Beyond "Hallucinations": A Framework for Stable Human-AI Reasoning](https://mlsanity.com/paper/2510.14665/) (2025)
- [Who Do LLMs Trust? Human Experts Matter More Than Other LLMs](https://mlsanity.com/paper/2602.13568/) (2026)
- [Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy](https://mlsanity.com/paper/2605.12991/) (2026)
- [The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Size](https://mlsanity.com/paper/2606.02646/) (2026)
- [When Agents "Misremember" Collectively: Exploring the Mandela Effect in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2602.00428/) (2026)
- [Governing Reflective Human-AI Collaboration: A Framework for Epistemic Scaffolding and Traceable Reasoning](https://mlsanity.com/paper/2604.14898/) (2026)
- [Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving](https://mlsanity.com/paper/2605.25393/) (2026)
- [Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems](https://mlsanity.com/paper/2605.27766/) (2026)
- [CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2605.29612/) (2026)
- [Conformity Mitigations in Large Language Models Lie on a Single Resistance-Receptivity Frontier](https://mlsanity.com/paper/2608.11247/) (2026)
- [LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems](https://mlsanity.com/paper/2609.33495/) (2026)
- [Assessing and Explaining the Persuadability of Large Language Models as Legal Decision Tools](https://mlsanity.com/paper/2604.26233/) (2026)
- [LUNA-AD: Lightweight Uncertainty-Aware Language Model with Lifelong Learning for Autonomous Driving](https://mlsanity.com/paper/2606.08470/) (2026)
- [ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents](https://mlsanity.com/paper/2607.16564/) (2026)
- [More Is Not More: What Matters for Diversity in LLM Opinions?](https://mlsanity.com/paper/2607.20429/) (2026)
- [When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA](https://mlsanity.com/paper/2609.08934/) (2026)
- [Unanimity Without Persuasion: A Single Round of Debate Erases the Disagreement That Verification Needs](https://mlsanity.com/paper/2609.26145/) (2026)
- [How does Adversarial Influence Scale in Multi-Agent Systems?](https://mlsanity.com/paper/2609.30028/) (2026)
- [BaRe-Mem: Bayesian Reliability Memory for Robust and Adaptive Agent Consultation](https://mlsanity.com/paper/2609.35551/) (2026)
- [Multi-agent discussion gains less when dissent is withheld](https://mlsanity.com/paper/2609.38324/) (2026)
- [When Order Matters: First-Speaker Bias and Mitigation through Personality in Sequential Multi-Agent Debate](https://mlsanity.com/paper/2609.38964/) (2026)

## Abstract (arXiv)

The conformity effect describes the tendency of individuals to align their responses with the majority. Studying this bias in large language models (LLMs) is crucial, as LLMs are increasingly used in various information-seeking and decision-making tasks as conversation partners to improve productivity. Thus, conformity to incorrect responses can compromise their effectiveness. In this paper, we adapt psychological experiments to examine the extent of conformity in popular LLMs. Our findings reveal that all tested models exhibit varying levels of conformity toward the majority, regardless of their initial choice or correctness, across different knowledge domains. Notably, we are the first to show that LLMs are more likely to conform when they are more uncertain in their own prediction. We further explore factors that influence conformity, such as training paradigms and input characteristics, finding that instruction-tuned models are less susceptible to conformity, while increasing the naturalness of majority tones amplifies conformity. Finally, we propose two interventions, Devil's Advocate and Question Distillation, to mitigate conformity, providing insights into building more robust language models.