mlsanityData to 7 Oct 2026

Do as We Do, Not as You Think: the Conformity of Large Language Models

arXiv 2501.13381 ↗First posted 23 Jan 2025cs.CLcited by 48

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Recent advancements in large language models (LLMs) revolutionize the field of intelligent agents, enabling collaborative multi-agent systems capable of tackling complex problems across various domains. However, the potential of conformity within these systems, analogous to phenomena like conformity bias and groupthink in human group dynamics, remains largely unexplored, raising concerns about their collective problem-solving capabilities and possible ethical implications. This paper presents a comprehensive study on conformity in LLM-driven multi-agent systems, focusing on three aspects: the existence of conformity, the factors influencing conformity, and potential mitigation strategies. In particular, we introduce BenchForm, a new conformity-oriented benchmark, featuring reasoning-intensive tasks and five distinct interaction protocols designed to probe LLMs' behavior in collaborative scenarios. Several representative LLMs are evaluated on BenchForm, using metrics such as conformity rate and independence rate to quantify conformity's impact. Our analysis delves into factors influencing conformity, including interaction time and majority size, and examines how the subject agent rationalizes its conforming behavior. Furthermore, we explore two strategies to mitigate conformity effects, i.e., developing enhanced personas and implementing a reflection mechanism. Several interesting findings regarding LLMs' conformity are derived from empirical results and case studies. We hope that these insights can pave the way for more robust and ethically-aligned collaborative AI systems. Our benchmark and code are available at BenchForm.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 34 of its 111 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

34 of 111 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System8 Dec
  2. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation2 Oct

Other references · 32

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  5. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  6. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  7. 2023Tool Learning with Foundation Modelscited by 302
  8. 2023AI Alignment: A Comprehensive Surveycited by 256
  9. 2023Self-collaboration Code Generation via ChatGPTcited by 255
  10. 2023Large Language Model Alignment: A Surveycited by 241
  11. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  12. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
20 more
  1. 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
  2. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  3. 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Warscited by 143
  4. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
  5. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  6. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84
  7. 2024What Evidence Do Language Models Find Convincing?cited by 74
  8. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  9. 2023When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Modelscited by 50
  10. 2023Epidemic Modeling with Generative Agentscited by 49
  11. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  12. 2023Welfare Diplomacy: Benchmarking Language Model Cooperationcited by 43
  13. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Frameworkcited by 33
  14. 2023AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisationcited by 31
  15. 2024Multi-Agent Collaboration via Cross-Team Orchestrationcited by 30
  16. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  17. 2024InvAgent: A Large Language Model based Multi-Agent System for Inventory Management in Supply Chainscited by 20
  18. 2023Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligencecited by 19
  19. 2024Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMscited by 4
  20. 2024Language Evolution for Evading Social Media Regulation via LLM-based Multi-agent Simulationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 31.

48 papers.

Sort:

By area:

  1. 2025Free-MAD: Consensus-Free Multi-Agent Debatecited by 27
  2. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 16
  3. 2025Herd Behavior: Investigating Peer Influence in LLM-based Multi-Agent Systemscited by 15
  4. 2025LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactionscited by 14
  5. 2025The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societiescited by 12
  6. 2025Do we really have to filter out random noise in pre-training data for language models?cited by 11
  7. 2026Conformity and Social Impact on AI Agentscited by 11
  8. 2025Many LLMs Are More Utilitarian Than Onecited by 7
  9. 2025Are Large Language Models Sensitive to the Motives Behind Communication?cited by 7
  10. 2026FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systemscited by 7
  11. 2026Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systemscited by 6
  12. 2026Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformitycited by 6
36 more
  1. 2025Disentangling the Drivers of LLM Social Conformity: An Uncertainty-Moderated Dual-Process Mechanismcited by 5
  2. 2025Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoningcited by 5
  3. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answeringcited by 5
  4. 2026Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systemscited by 5
  5. 2025LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertaintycited by 4
  6. 2025MADRA: Multi-Agent Debate for Risk-Aware Embodied Planningcited by 4
  7. 2026Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarkscited by 4
  8. 2025Towards Simulating Social Influence Dynamics with LLM-based Multi-agentscited by 3
  9. 2025Spiral of Silence in Large Language Model Agentscited by 3
  10. 2025The Effect of Belief Boxes and Open-mindedness on Persuasioncited by 3
  11. 2026Who Do LLMs Trust? Human Experts Matter More Than Other LLMscited by 3
  12. 2026When Agents "Misremember" Collectively: Exploring the Mandela Effect in LLM-based Multi-Agent Systemscited by 2
  13. 2026Agentic Microphysics: A Manifesto for Generative AI Safetycited by 2
  14. 2026The Reasoning Trap: An Information-Theoretic Bound on Closed-System Multi-Step LLM Reasoningcited by 2
  15. 2025Multi-Agent Synergy-Driven Iterative Visual Narrative Synthesiscited by 1
  16. 2025Belief-Calibrated Multi-Agent Consensus Seeking for Complex NLP Taskscited by 1
  17. 2026SinkTrack: Attention Sink based Context Anchoring for Large Language Modelscited by 1
  18. 2026What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debatescited by 1
  19. 2026Social Pressure Breaks Majority Voting in LLM Safety Panelscited by 1
  20. 2026MEDLEY-BENCH: Benchmarking Behavioural Metacognition and Belief Revision Under Social Pressure in Large Language Modelscited by 0
  21. 2026AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimizationcited by 0
  22. 2026PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Predictioncited by 0
  23. 2026TRACE: An Operational Reasoning Schema for Auditable Agentic Commitmentscited by 0
  24. 2026Where Reliability Lives: Experimental Localisation of Behavioural Properties in an Agent Systemcited by 0
  25. 2026The Civilization Framework: Sovereign-Anchored Communication Between Personal Multi-Agent Systemscited by 0
  26. 2026Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systemscited by 0
  27. 2026A Dual-Process Perspective on Nudge Susceptibility in LLM-Based GUI Agentscited by 0
  28. 2026Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agentscited by 0
  29. 2026Emergent Collusion in Long-Horizon LLM Agent Interactioncited by 0
  30. 2026How does Adversarial Influence Scale in Multi-Agent Systems?cited by 0
  31. 2026Multi-agent Scaling Across Disjunctive and Compensatory Taskscited by 0
  32. 2026LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systemscited by 0
  33. 2026BaRe-Mem: Bayesian Reliability Memory for Robust and Adaptive Agent Consultationcited by 0
  34. 2026Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeablecited by 0
  35. 2026Multi-agent discussion gains less when dissent is withheldcited by 0
  36. 2026When Order Matters: First-Speaker Bias and Mitigation through Personality in Sequential Multi-Agent Debatecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Large Language Models Exhibit Normative Conformity
  2. 2026Conformity Dynamics in LLM Multi-Agent Systems: The Roles of Topology and Self-Social Weighting
  3. 2025An Empirical Study of Group Conformity in Multi-Agent Systems
  4. 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes
  5. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
  6. 2025Large Language Models Miss the Multi-Agent Mark
  7. 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation
  8. 2025Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspective