# DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents

arXiv: https://arxiv.org/abs/2510.25110 · HTML: https://mlsanity.com/paper/2510.25110/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 20 of 20)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions](https://mlsanity.com/paper/1602.01103/) (2016)
- [Systematic Biases in LLM Simulations of Debates](https://mlsanity.com/paper/2402.04049/) (2024)
- [From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News](https://mlsanity.com/paper/2403.09498/) (2024)
- [Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks](https://mlsanity.com/paper/2406.17232/) (2024)
- [The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents](https://mlsanity.com/paper/2311.09665/) (2023)
- [Debate-to-Write: A Persona-Driven Multi-Agent Framework for Diverse Argument Generation](https://mlsanity.com/paper/2406.19643/) (2024)
- [Conversational flow in Oxford-style debates](https://mlsanity.com/paper/1604.03114/) (2016)
- [Computational Agent-based Models in Opinion Dynamics: A Survey on Social Simulations and Empirical Studies](https://mlsanity.com/paper/2306.03446/) (2023)
- [Asking Too Much? The Rhetorical Role of Questions in Political Discourse](https://mlsanity.com/paper/1708.02254/) (2017)
- [From Competition to Complementarity: Comparative Influence Diffusion and Maximization](https://mlsanity.com/paper/1507.00317/) (2015)

## Built on by (most cited first, 6 of 6)

- [Can LLMs Think Like Consumers? Benchmarking Crowd-Level Reaction Reconstruction with ConsumerSimBench](https://mlsanity.com/paper/2605.17079/) (2026)
- [Convergence of Outputs When Two Large Language Models Interact in a Multi-Agentic Setup](https://mlsanity.com/paper/2512.06256/) (2025)
- [CHAL: Council of Hierarchical Agentic Language](https://mlsanity.com/paper/2605.12718/) (2026)
- [Do LLM Debates Repeat Arguments Differently Across Languages?](https://mlsanity.com/paper/2607.23442/) (2026)
- [From Memory to Behavior: A Behavior-Aware Role-Playing Framework for Social Media Influencers](https://mlsanity.com/paper/2609.21349/) (2026)
- [Do AI Agents Gossip? Evaluative Talk in Human and AI-Agent Online Communities](https://mlsanity.com/paper/2610.20360/) (2026)

## Abstract (arXiv)

Accurately modeling opinion change through social interactions is crucial for understanding and mitigating polarization, misinformation, and societal conflict. Recent work simulates opinion dynamics with role-playing LLM agents (RPLAs), but multi-agent simulations often display unnatural group behavior, such as premature convergence, and lack empirical benchmarks for assessing alignment with real human group interactions. We introduce DEBATE, a large-scale benchmark for evaluating the authenticity of opinion dynamics in multi-agent RPLA simulations. DEBATE contains multi-round public messages and private Likert-scale beliefs from U.S.-based participants across 107 topics; the cleaned benchmark used in our experiments contains 2,788 participants in 697 groups, enabling evaluation at the utterance and group levels and supporting future individual-level analyses. We instantiate "digital twin" RPLAs with seven LLMs and evaluate across two settings: next-message prediction and full dynamics simulation, using stance-based opinion-dynamics metrics. In zero-shot settings, RPLA groups exhibit strong opinion convergence relative to human groups. On the held-out group split, supervised fine-tuning (SFT) for Llama-3.1-8B-Instruct improves auxiliary stance alignment and reduces group-level convergence error, though discrepancies in opinion change and belief updating remain. DEBATE enables rigorous benchmarking of simulated opinion dynamics and supports future research on aligning multi-agent RPLAs with realistic human interactions.