IO Factory: Simulating AI-Enabled Influence Campaigns at Scale
Not on the map. Why?
Abstract (arXiv)
We introduce IO Factory, an AI-driven framework for simulating information and influence campaigns as fully integrated, traceable processes. The threat of digital manipulation now extends beyond persuasive text from individual language models to AI swarms, i.e., persistent groups of coordinated agents that adapt to platform feedback and disguise organized campaigns as ordinary social interaction. Because such campaigns cannot be identified from isolated messages alone, they must be analyzed across a continuous spectrum of planning, platform action, exposure, interpretation, measurement, and adaptation. IO Factory represents this process inside a controlled simulated platform, linking actor roles, platform actions, exposure records, structured model-based evaluations, and configured changes in the simulated population. We implement the architecture and evaluate it across configurations of up to 100,000 agents. The results show that IO Factory executes campaign timelines at scale and produces inspectable evidence of exposure and measured movement in configured belief variables. By recording the actors, objectives, action constraints, exposure paths, and measurement rules used in each run, IO Factory supports reproducible research and red-team analysis of coordinated influence.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 92 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
27 of 92 references.
References · 27
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2018Datasheets for Datasetscited by 1,386
- 2018Model Cards for Model Reportingcited by 1,080
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
- 2019Release Strategies and the Social Impacts of Language Modelscited by 513
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
- 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
15 more
- 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 203
- 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 165
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
- 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 154
- 2024OASIS: Open Agent Social Interaction Simulations with One Million Agentscited by 132
- 2014The Rise of Social Botscited by 96
- 2024GenSim: A General Social Simulation Platform with Large Language Model based Agentscited by 50
- 2020Uncovering Coordinated Networks on Social Media: Methods and Case Studiescited by 49
- 2025How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfarecited by 20
- 2023Computational Agent-based Models in Opinion Dynamics: A Survey on Social Simulations and Empirical Studiescited by 9
- 2023Temporal Dynamics of Coordinated Online Behavior: Stability, Archetypes, and Influencecited by 6
- 2024Detection and Characterization of Coordinated Online Behavior: A Surveycited by 6
- 2025Emergent Coordinated Behaviors in Networked LLM Agents: Modeling the Strategic Dynamics of Information Operationscited by 6
- 2025AI Propaganda factories with language modelscited by 4
- 2026Puppets or partners? Governing cyborg propaganda in the digital public squarecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Execution and assessment of agentic influence operations in simulated social networks
- 2024Synthetic Social Media Influence Experimentation via an Agentic Reinforcement Learning Large Language Model Bot
- 2026ElecTwit: A Framework for Studying Persuasion in Multi-Agent Social Systems
- 2026Modeling Social Dynamics with an LLM-Enabled Agent Based Network-Dynamic (LAND) Model
- 2024Agent-Based Modelling Meets Generative AI in Social Network Simulations
- 2026Knowing You Is Everything: LLM Agents Achieve Near-Perfect Profile-Consistent Reaction Prediction in Social Media Simulation
- 2026Conformity and Social Impact on AI Agents
- 2026Behavioral Determinants of Deployed AI Agents in Social Networks: A Multi-Factor Study of Personality, Model, and Guardrail Specification