# A Simulation System Towards Solving Societal-Scale Manipulation

arXiv: https://arxiv.org/abs/2410.13915 · HTML: https://mlsanity.com/paper/2410.13915/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial](https://mlsanity.com/paper/2403.14380/) (2024)
- [The Rise of Social Bots](https://mlsanity.com/paper/1407.5225/) (2014)
- [The Political Preferences of LLMs](https://mlsanity.com/paper/2402.01789/) (2024)
- [Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation](https://mlsanity.com/paper/2402.16333/) (2024)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [On the steerability of large language models toward data-driven personas](https://mlsanity.com/paper/2311.04978/) (2023)
- [Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values](https://mlsanity.com/paper/2311.10766/) (2023)
- [Assessing Political Bias in Large Language Models](https://mlsanity.com/paper/2405.13041/) (2024)
- [Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments](https://mlsanity.com/paper/2404.09329/) (2024)
- [TwHIN: Embedding the Twitter Heterogeneous Information Network for Personalized Recommendation](https://mlsanity.com/paper/2202.05387/) (2022)
- [The Effect of People Recommenders on Echo Chambers and Polarization](https://mlsanity.com/paper/2112.00626/) (2021)
- [Disinformation Warfare: Understanding State-Sponsored Trolls on Twitter and Their Influence on the Web](https://mlsanity.com/paper/1801.09288/) (2018)
- [Modeling echo chambers and polarization dynamics in social networks](https://mlsanity.com/paper/1906.12325/) (2019)
- [Are Large Language Models (LLMs) Good Social Predictors?](https://mlsanity.com/paper/2402.12620/) (2024)
- [Modeling confirmation bias and polarization](https://mlsanity.com/paper/1607.00022/) (2016)
- [Whose Side Are You On? Investigating the Political Stance of Large Language Models](https://mlsanity.com/paper/2403.13840/) (2024)
- [Capitol (Pat)riots: A comparative study of Twitter and Parler](https://mlsanity.com/paper/2101.06914/) (2021)
- [Echo chambers in the age of misinformation](https://mlsanity.com/paper/1509.00189/) (2015)
- [Algorithmic bias amplifies opinion polarization: A bounded confidence model](https://mlsanity.com/paper/1803.02111/) (2018)
- [Opinion Dynamics with Backfire Effect and Biased Assimilation](https://mlsanity.com/paper/1903.11535/) (2019)
- [The effect of algorithmic bias and network structure on coexistence, consensus, and polarization of opinions](https://mlsanity.com/paper/2105.07703/) (2021)
- [The role of homophily in the emergence of opinion controversies](https://mlsanity.com/paper/1612.05483/) (2016)

## Built on by (most cited first, 12 of 12)

- [From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents](https://mlsanity.com/paper/2412.03563/) (2024)
- [Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy](https://mlsanity.com/paper/2503.09639/) (2025)
- [The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies](https://mlsanity.com/paper/2509.18052/) (2025)
- [textttBluePrint: A Social Media User Dataset for LLM Persona Evaluation and Training](https://mlsanity.com/paper/2510.02343/) (2025)
- [NegotiationGym: Self-Optimizing Agents in a Multi-Agent Social Simulation Environment](https://mlsanity.com/paper/2510.04368/) (2025)
- [SimCity: Multi-Agent Urban Development Simulation with Rich Interactions](https://mlsanity.com/paper/2510.01297/) (2025)
- [Synthia: Scalable Grounded Persona Generation from Social Media Data](https://mlsanity.com/paper/2507.14922/) (2025)
- [Inject, Fork, Compare: Defining an Interaction Vocabulary for Multi-Agent Simulation Platforms](https://mlsanity.com/paper/2509.13712/) (2025)
- [Prototyping Digital Social Spaces through Metaphor-Driven Design: Translating Spatial Concepts into an Interactive Social Simulation](https://mlsanity.com/paper/2510.02759/) (2025)
- [Predicting Social Media User Actions: A Hybrid Approach for Common and Rare Behavior Prediction on Bluesky](https://mlsanity.com/paper/2511.17241/) (2025)
- [From Who They Are to How They Act: Behavioral Traits in Generative Agent-Based Models of Social Media](https://mlsanity.com/paper/2601.15114/) (2026)
- [Eco3S: Complex Socio-Economic System Simulation via Agent-Based Models](https://mlsanity.com/paper/2607.26588/) (2026)

## Abstract (arXiv)

The rise of AI-driven manipulation poses significant risks to societal trust and democratic processes. Yet, studying these effects in real-world settings at scale is ethically and logistically impractical, highlighting a need for simulation tools that can model these dynamics in controlled settings to enable experimentation with possible defenses. We present a simulation environment designed to address this. We elaborate upon the Concordia framework that simulates offline, `real life' activity by adding online interactions to the simulation through social media with the integration of a Mastodon server. We improve simulation efficiency and information flow, and add a set of measurement tools, particularly longitudinal surveys. We demonstrate the simulator with a tailored example in which we track agents' political positions and show how partisan manipulation of agents can affect election results.