mlsanityData to 7 Oct 2026

Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks

arXiv 2607.05545 ↗First posted 6 Jul 2026cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

LLM conformity is often used to describe cases where a model changes a correct answer toward a peer or group response. We show that most of this apparent conformity survives even after the peer is removed. The reason is a confound: standard conformity prompts mix two cues at once, the presence of a speaker and the repeated wrong answer itself. Existing benchmarks vary these cues together, so they cannot tell how much of the revision actually depends on the speaker. We introduce a no-source condition: the same asserted answer with the explicit speaker removed. Across six open-weight LLMs and seven QA and reasoning datasets, this condition alone causes harmful revision in 66.5% of initially correct cases, compared with 10.3% under a plain re-ask. The effect also remains when the repeated answer is paraphrased and when answer options are hidden in an open-ended setting. Source framing mainly modulates this floor: expert-panel framing raises it, while minimal person labels do not reliably raise it. When models flip, they are usually confidently wrong, and simple recalibration does not recover the original answer. Source attribution still matters, but it should be measured as an increment above this speaker-free floor. The methodological lesson is that conformity benchmarks should first measure what remains after the speaker is removed; without this step, benchmarks may mistake repeated text for social influence.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 45 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

25 of 45 references.

References · 25

  1. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  7. 2024Emergent social conventions and collective bias in LLM populationscited by 115
  8. 2024Are self-explanations from Large Language Models faithful?cited by 101
  9. 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 71
  10. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 48
  11. 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 32
  12. 2025TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Modelscited by 31
13 more
  1. 2025DarkBench: Benchmarking Dark Patterns in Large Language Modelscited by 25
  2. 2025Interaction Context Often Increases Sycophancy in LLMscited by 21
  3. 2024When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settingscited by 17
  4. 2025LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactionscited by 14
  5. 2025When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMscited by 14
  6. 2026Whose Facts Win? LLM Source Preferences under Knowledge Conflictscited by 8
  7. 2026Conformity Dynamics in LLM Multi-Agent Systems: The Roles of Topology and Self-Social Weightingcited by 8
  8. 2026Conformity Generates Collective Misalignment in AI Agents Societiescited by 8
  9. 2026Old Habits Die Hard: How Conversational History Geometrically Traps LLMscited by 5
  10. 2026Large Language Models Exhibit Normative Conformitycited by 5
  11. 2026Belief in Authority: Impact of Authority in Multi-Agent Evaluation Frameworkcited by 4
  12. 2022Controllable Fake Document Infilling for Cyber Deceptioncited by 3
  13. 2026Who Do LLMs Trust? Human Experts Matter More Than Other LLMscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity
  2. 2026Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding
  3. 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
  4. 2026Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
  5. 2026One Axis, No Brake: Self-Knowledge Limits the Filtering of Harmful Peer Conformity in LLMs
  6. 2026Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
  7. 2026Feedback Indices to Evaluate LLM Responses to Rebuttals for Multiple Choice Type Questions
  8. 2026Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective