I Want to Break Free! Persuasion and Anti-Social Behavior of LLMs in Multi-Agent Settings with Social Hierarchy
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
As LLM-based agents become increasingly autonomous and will more freely interact with each other, studying the interplay among them becomes crucial to anticipate emergent phenomena and potential risks. In this work, we provide an in-depth analysis of the interactions among agents within a simulated hierarchical social environment, drawing inspiration from the Stanford Prison Experiment. Leveraging 2,400 conversations across six LLMs (i.e., LLama3, Orca2, Command-r, Mixtral, Mistral2, and gpt4.1) and 240 experimental scenarios, we analyze persuasion and anti-social behavior between a guard and a prisoner agent with differing objectives. We first document model-specific conversational failures in this multi-agent power dynamic context, thereby narrowing our analytic sample to 1,600 conversations. Among models demonstrating successful interaction, we find that goal setting significantly influences persuasiveness but not anti-social behavior. Moreover, agent personas, especially the guard's, substantially impact both successful persuasion by the prisoner and the manifestation of anti-social actions. Notably, we observe the emergence of anti-social conduct even in absence of explicit negative personality prompts. These results have important implications for the development of interactive LLM agents and the ongoing discussion of their societal impact.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 16 of its 70 references that are in the map. First posted on arXiv 9 Oct 2024.
Builds on
16 of 70 references.
Closest ancestors · 1
Same direction, newest first
Other references · 15
- 2023GPT-4 Technical Reportcited by 18,396
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Mistral 7Bcited by 3,164
- 2024Mixtral of Expertscited by 1,823
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2020Open Problems in Cooperative AIcited by 168
- 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
- 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 120
- 2024On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trialcited by 109
- 2023Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithmscited by 65
Built on by
Citing papers per year: 2025: 5; 2026: 7.
12 papers.
- 2025Agentic Large Language Models, a surveycited by 108
- 2025Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agentscited by 22
- 2026Does Socialization Emerge in AI Agent Society? A Case Study of Moltbookcited by 15
- 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 13
- 2025Agentic LLM Framework for Adaptive Decision Discoursecited by 6
- 2026Generative AI collective behavior needs an interactionist paradigmcited by 6
- 2025MedSyn: Enhancing Diagnostics with Human-AI Collaborationcited by 2
- 2026Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experimentcited by 2
- 2026Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agentscited by 1
- 2026Evaluating Large Language Models in a Complex Hidden Role Gamecited by 1
- 2026Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modelingcited by 0
- 2026PAVE: A Cognitive Architecture for Legitimate Violation in Generative Agent Societiescited by 0
- 2026Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experiment20 May 2026
- 2026PAVE: A Cognitive Architecture for Legitimate Violation in Generative Agent Societies19 May 2026
- 2026Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling12 May 2026
- 2026Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents24 Apr 2026
- 2026Evaluating Large Language Models in a Complex Hidden Role Game9 Apr 2026
- 2026Does Socialization Emerge in AI Agent Society? A Case Study of Moltbook15 Feb 2026
- 2026Generative AI collective behavior needs an interactionist paradigm15 Jan 2026
- 2025MedSyn: Enhancing Diagnostics with Human-AI Collaboration7 May 2025
- 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective4 Apr 2025
- 2025Agentic Large Language Models, a survey29 Mar 2025
- 2025Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents18 Feb 2025
- 2025Agentic LLM Framework for Adaptive Decision Discourse16 Feb 2025
Similar, not cited
Close in wording, no citation either way
- 2023Do LLM Agents Exhibit Social Behavior?
- 2024Evolution of Social Norms in LLM Agents using Natural Language
- 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation
- 2026Collective cooperation without individual fidelity in LLM agents
- 2025Learning to Make Friends: Coaching LLM Agents toward Emergent Social Ties
- 2026Behavioral Determinants of Deployed AI Agents in Social Networks: A Multi-Factor Study of Personality, Model, and Guardrail Specification
- 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
- 2025LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions