# IO Factory: Simulating AI-Enabled Influence Campaigns at Scale

arXiv: https://arxiv.org/abs/2608.10920 · HTML: https://mlsanity.com/paper/2608.10920/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 92 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Builds on (most cited first, 27 of 27)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Poisoning Web-Scale Training Datasets is Practical](https://mlsanity.com/paper/2302.10149/) (2023)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [Poisoning Retrieval Corpora by Injecting Adversarial Passages](https://mlsanity.com/paper/2310.19156/) (2023)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [OASIS: Open Agent Social Interaction Simulations with One Million Agents](https://mlsanity.com/paper/2411.11581/) (2024)
- [The Rise of Social Bots](https://mlsanity.com/paper/1407.5225/) (2014)
- [GenSim: A General Social Simulation Platform with Large Language Model based Agents](https://mlsanity.com/paper/2410.04360/) (2024)
- [Uncovering Coordinated Networks on Social Media: Methods and Case Studies](https://mlsanity.com/paper/2001.05658/) (2020)
- [How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfare](https://mlsanity.com/paper/2506.06299/) (2025)
- [Computational Agent-based Models in Opinion Dynamics: A Survey on Social Simulations and Empirical Studies](https://mlsanity.com/paper/2306.03446/) (2023)
- [Temporal Dynamics of Coordinated Online Behavior: Stability, Archetypes, and Influence](https://mlsanity.com/paper/2301.06774/) (2023)
- [Detection and Characterization of Coordinated Online Behavior: A Survey](https://mlsanity.com/paper/2408.01257/) (2024)
- [Emergent Coordinated Behaviors in Networked LLM Agents: Modeling the Strategic Dynamics of Information Operations](https://mlsanity.com/paper/2510.25003/) (2025)
- [AI Propaganda factories with language models](https://mlsanity.com/paper/2508.20186/) (2025)
- [Puppets or partners? Governing cyborg propaganda in the digital public square](https://mlsanity.com/paper/2602.13088/) (2026)

## Abstract (arXiv)

We introduce IO Factory, an AI-driven framework for simulating information and influence campaigns as fully integrated, traceable processes. The threat of digital manipulation now extends beyond persuasive text from individual language models to AI swarms, i.e., persistent groups of coordinated agents that adapt to platform feedback and disguise organized campaigns as ordinary social interaction. Because such campaigns cannot be identified from isolated messages alone, they must be analyzed across a continuous spectrum of planning, platform action, exposure, interpretation, measurement, and adaptation. IO Factory represents this process inside a controlled simulated platform, linking actor roles, platform actions, exposure records, structured model-based evaluations, and configured changes in the simulated population. We implement the architecture and evaluate it across configurations of up to 100,000 agents. The results show that IO Factory executes campaign timelines at scale and produces inspectable evidence of exposure and measured movement in configured belief variables. By recording the actors, objectives, action constraints, exposure paths, and measurement rules used in each run, IO Factory supports reproducible research and red-team analysis of coordinated influence.