mlsanityData to 14 Oct 2026

Social Pressure Breaks Majority Voting in LLM Safety Panels

arXiv 2608.04415 ↗First posted 5 Aug 2026cs.CLcited by 1

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large language models (LLMs) are increasingly used to detect unsafe content. A common approach is to combine judgments from a panel of models to correct individual mistakes, but this benefit may disappear when every model sees the same misleading context before voting. We study this risk in a controlled two-round experiment. Each model first judges an item alone, then judges it again after six simulated peers either assert the wrong label or abstain. We combine the final judgments by majority vote. Across six open-weight LLMs and six datasets, we find that the wrong-label peer message raises the average reviewer false-alarm rate from 56.5% under silent peers to 87.5%, and majority voting raises the panel false-alarm rate to 100%. Without an asserted label, the same panel outperforms its average member. The effect is strongly asymmetric: reviewers follow pushes toward "unsafe" far more than pushes toward "safe" (about 75% versus 17%), so the panel's false-alarm rate rises sharply while its harmful-miss rate changes little. The proprietary-model probe shows substantial variation across models. These results identify susceptibility to shared social cues as a failure mode of safety panels and provide a simple pre-deployment diagnostic.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 21 of its 38 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

21 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs26 Jun

Other references · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  6. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  7. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  8. 2020Aligning AI With Shared Human Valuescited by 723
  9. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  10. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
  11. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
  12. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 251
8 more
  1. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
  2. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  3. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 53
  4. 2025When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMscited by 16
  5. 2026Conformity Dynamics in LLM Multi-Agent Systems: The Roles of Topology and Self-Social Weightingcited by 10
  6. 2026Conformity Generates Collective Misalignment in AI Agents Societiescited by 9
  7. 2026Belief in Authority: Impact of Authority in Multi-Agent Evaluation Frameworkcited by 6
  8. 2026Large Language Models Exhibit Normative Conformitycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels
  2. 2026Forged Peer Judgments Mislead Multimodal LLM Judge Panels: Source-Blind Anchoring and Panel-Consensus Verification
  3. 2026Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity
  4. 2025Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts
  5. 2025SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests
  6. 2025LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models
  7. 2025Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yet
  8. 2025LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language