# Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits

arXiv: https://arxiv.org/abs/2605.18890 · HTML: https://mlsanity.com/paper/2605.18890/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 73 references that are in the map.
- First posted on arXiv 17 May 2026.

## Builds on (most cited first, 30 of 46)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society](https://mlsanity.com/paper/2502.08691/) (2025)
- [EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities](https://mlsanity.com/paper/2310.10436/) (2023)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance](https://mlsanity.com/paper/2401.03729/) (2024)
- [Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation](https://mlsanity.com/paper/2309.17234/) (2023)
- [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025)
- [Humanoid Agents: Platform for Simulating Human-like Generative Agents](https://mlsanity.com/paper/2310.05418/) (2023)
- [Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation](https://mlsanity.com/paper/2402.16333/) (2024)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?](https://mlsanity.com/paper/2406.13605/) (2024)
- [Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing](https://mlsanity.com/paper/2309.05898/) (2023)
- [From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News](https://mlsanity.com/paper/2403.09498/) (2024)
- [TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets](https://mlsanity.com/paper/2502.01506/) (2025)
- [Emergence of Social Norms in Generative Agent Societies: Principles and Architecture](https://mlsanity.com/paper/2403.08251/) (2024)
- [Benchmarking Prompt Sensitivity in Large Language Models](https://mlsanity.com/paper/2502.06065/) (2025)
- [Decoding Echo Chambers: LLM-Powered Simulations Revealing Polarization in Social Networks](https://mlsanity.com/paper/2409.19338/) (2024)
- [The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents](https://mlsanity.com/paper/2311.09665/) (2023)

## Built on by (most cited first, 8 of 8)

- [Mitigating LLM-based p-Hacking by Preregistering for the Next LLM](https://mlsanity.com/paper/2606.27687/) (2026)
- [Belief Coevolution in a Social Network of Generalist and Specialist Large Language Models](https://mlsanity.com/paper/2607.27512/) (2026)
- [Defeat Devices in AI Systems](https://mlsanity.com/paper/2606.28863/) (2026)
- [Collective cooperation without individual fidelity in LLM agents](https://mlsanity.com/paper/2606.30454/) (2026)
- [Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation](https://mlsanity.com/paper/2608.20569/) (2026)
- [Benchmarking large language model agent societies against human behavioural distributions](https://mlsanity.com/paper/2608.28182/) (2026)
- [Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga](https://mlsanity.com/paper/2610.13840/) (2026)
- [From Caricature to Collective Artifact: How Persona Construction Drives Polarization in Generative Social Simulations](https://mlsanity.com/paper/2610.16498/) (2026)

## Abstract (arXiv)

The scientific claims drawn from LLM social simulations should be no stronger than the robustness audits that support them. Generative agents bring new expressive power to agent-based modeling, enabling simulations of collective social processes like cooperation, polarization, and norm formation. Yet they also introduce complexity through additional architectural choices, such as agent specification, memory representation, interaction protocols, and environment design. Small perturbations that appear minor to researchers can cascade into macro-level outcomes through repeated interaction, creating a "butterfly effect." Consequently, scientific claims drawn from LLM social simulations may reflect implementation artifacts rather than the social mechanisms being modeled. We support this position with two case studies: a repeated Prisoner's Dilemma and a social media echo chamber simulation. Across multiple models, minor perturbations in persona format and game-instruction framing shift cooperation rates by up to 76 percentage points, while network homophily and hub assignment produce significant and consistent shifts in polarization metrics. We also find that sensitivity is unevenly distributed across both architectural choices and model families: the same perturbation that produces the 76 pp shift in one frontier model only shifts another by 1 pp. Robustness is therefore a property that should be measured per claim and per model, not assumed. To address this validation gap, we introduce TRAILS (Taxonomy for Robustness Audits In LLM Simulations), a robustness-audit taxonomy spanning three levels of simulation design: agent (micro-level), interaction (meso-level), and system (macro-level). We call for robustness to become a first-order validation requirement before LLM social simulations are used to explain mechanisms, evaluate interventions, or inform decisions.