# How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfare

arXiv: https://arxiv.org/abs/2506.06299 · HTML: https://mlsanity.com/paper/2506.06299/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 92 references that are in the map.
- First posted on arXiv 18 May 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 26 of 26)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [Machine Culture](https://mlsanity.com/paper/2311.11388/) (2023)
- [Uncovering Coordinated Networks on Social Media: Methods and Case Studies](https://mlsanity.com/paper/2001.05658/) (2020)
- [Anatomy of an AI-powered malicious social botnet](https://mlsanity.com/paper/2307.16336/) (2023)
- [Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Media](https://mlsanity.com/paper/2305.16941/) (2023)
- [Personhood credentials: Artificial intelligence and the value of privacy-preserving tools to distinguish who is real online](https://mlsanity.com/paper/2408.07892/) (2024)
- [Characteristics and prevalence of fake social media profiles with AI-generated faces](https://mlsanity.com/paper/2401.02627/) (2024)
- [Large language models can consistently generate high-quality content for election disinformation operations](https://mlsanity.com/paper/2408.06731/) (2024)
- [Evolving AI Collectives to Enhance Human Diversity and Enable Self-Regulation](https://mlsanity.com/paper/2402.12590/) (2024)
- [From alternative conceptions of honesty to alternative facts in communications by U.S. politicians](https://mlsanity.com/paper/2208.10814/) (2022)
- [Shaping New Norms for AI](https://mlsanity.com/paper/2307.08564/) (2023)
- [Quantifying the Vulnerabilities of the Online Public Square to Adversarial Manipulation Tactics](https://mlsanity.com/paper/1907.06130/) (2019)
- [How will advanced AI systems impact democracy?](https://mlsanity.com/paper/2409.06729/) (2024)
- [Crisis, Country, and Party Lines: Politicians' Misinformation Behavior and Public Engagement](https://mlsanity.com/paper/2502.15321/) (2025)

## Built on by (most cited first, 20 of 20)

- [Group size effects and collective misalignment in LLM multi-agent systems](https://mlsanity.com/paper/2510.22422/) (2025)
- [Shaping capabilities with token-level data filtering](https://mlsanity.com/paper/2601.21571/) (2026)
- [Collective Behavior of AI Agents: the Case of Moltbook](https://mlsanity.com/paper/2602.09270/) (2026)
- [Emergent Coordinated Behaviors in Networked LLM Agents: Modeling the Strategic Dynamics of Information Operations](https://mlsanity.com/paper/2510.25003/) (2025)
- [AI Propaganda factories with language models](https://mlsanity.com/paper/2508.20186/) (2025)
- [Machines in the Crowd? Measuring the Footprint of Machine-Generated Text on Reddit](https://mlsanity.com/paper/2510.07226/) (2025)
- [Reply to "Emergent LLM behaviors are observationally equivalent to data leakage"](https://mlsanity.com/paper/2506.18600/) (2025)
- [The Collective Turing Test: Large Language Models Can Generate Realistic Multi-User Discussions](https://mlsanity.com/paper/2511.08592/) (2025)
- [Copying explains the collective behavior of AI agents in the wild](https://mlsanity.com/paper/2609.09150/) (2026)
- [The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents](https://mlsanity.com/paper/2601.12164/) (2026)
- [Indirect tipping: a social attack surface in AI agent populations](https://mlsanity.com/paper/2609.25194/) (2026)
- [Artificially intelligent agents in the social and behavioral sciences: A history and outlook](https://mlsanity.com/paper/2510.05743/) (2025)
- [How Far Will They Go? Red-Teaming Online Influence with Large Language Models](https://mlsanity.com/paper/2605.22880/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [Adversarial Creation and Detection of AI-Generated Social Bot Content](https://mlsanity.com/paper/2606.07219/) (2026)
- [Cross-National Information Attacks: A Two-Decade Analysis of Troll Behavior in Korea](https://mlsanity.com/paper/2606.22785/) (2026)
- [IO Factory: Simulating AI-Enabled Influence Campaigns at Scale](https://mlsanity.com/paper/2608.10920/) (2026)
- [The AI Accountability Ecosystem in the Era of Language Models](https://mlsanity.com/paper/2608.12320/) (2026)
- [Belief Cascades Drive Persuasion in LLM Agent Networks](https://mlsanity.com/paper/2608.25152/) (2026)
- [AI agents reshape consensus formation in human groups](https://mlsanity.com/paper/2609.02122/) (2026)

## Abstract (arXiv)

Advances in AI offer the prospect of manipulating beliefs and behaviors on a population-wide level. Large language models and autonomous agents now let influence campaigns reach unprecedented scale and precision. Generative tools can expand propaganda output without sacrificing credibility and inexpensively create falsehoods that are rated as more human-like than those written by humans. Techniques meant to refine AI reasoning, such as chain-of-thought prompting, can just as effectively be used to generate more convincing falsehoods. Enabled by these capabilities, a disruptive threat is emerging: swarms of collaborative, malicious AI agents. Fusing LLM reasoning with multi-agent architectures, these systems are capable of coordinating autonomously, infiltrating communities, and fabricating consensus efficiently. By adaptively mimicking human social dynamics, they threaten democracy. Because the resulting harms stem from design, commercial incentives, and governance, we prioritize interventions at multiple leverage points, focusing on pragmatic mechanisms over voluntary compliance.