Indirect tipping: a social attack surface in AI agent populations
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
As generative AI agents are deployed at scale, safety will depend not only on technical safeguards and individual model design, but also on collective equilibria that determine how agent populations process information, prioritize actions, and respond to uncertainty. Yet the same equilibria that enable agents to coordinate also create a social attack surface. The standard framework to assess this vulnerability is critical mass dynamics: the minimum fraction of adversarial agents required to overturn an equilibrium through direct competition. Here, we show that this approach risks underestimating system vulnerability by reducing the problem to the identification of singular tipping points, and ignoring indirect but potentially more efficient routes through which collective behavior can be redirected. Through experiments with populations of LLM agents and an analytic framework that captures their collective dynamics at scale, we map critical-mass thresholds that define a directed, weighted topology over the space of coordination equilibria, and treat this topology as a navigable landscape. We show that indirect tipping through intermediate stepping-stone equilibria can reduce the committed minority required to reach an alternative state, bypass majority requirements, and make possible transitions inaccessible through direct challenges. The diversity of available alternatives and timing of the attack further reshape this landscape, creating opportunities for control as well as risks of unintended destabilization. These results show that an equilibrium's resistance to committed intervention is not an intrinsic property but a structural feature of its competitive relations with alternative states. Securing populations of interacting AI agents therefore requires mapping this social landscape alongside individual agent capabilities and the technical channels through which they interact.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 30 of its 71 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
30 of 71 references.
Closest ancestors · 1
Same direction, newest first
Other references · 29
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
- 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
- 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
- 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
- 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
- 2025Multi-Agent Risks from Advanced AIcited by 163
- 2024Emergent social conventions and collective bias in LLM populationscited by 123
- 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
17 more
- 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 63
- 2024Algorithmic Collusion by Large Language Modelscited by 62
- 2023Machine Culturecited by 55
- 2023Anatomy of an AI-powered malicious social botnetcited by 36
- 2024A new sociology of humans and machinescited by 32
- 2021Reinforcement Learning for Quantitative Tradingcited by 24
- 2015The Spontaneous Emergence of Conventions: An Experimental Study of Cultural Evolutioncited by 17
- 2025Group size effects and collective misalignment in LLM multi-agent systemscited by 16
- 2017The Emergence of Consensus: A Primercited by 12
- 2025AI with Emotions: Exploring Emotional Expressions in Large Language Modelscited by 11
- 2026The Rise of AI Agent Communities: Large-Scale Analysis of Discourse and Interaction on Moltbookcited by 9
- 2018The Dynamics of Norm Change in the Cultural Evolution of Languagecited by 8
- 2026OpenClaw AI Agents as Informal Learners at Moltbook: Characterizing an Emergent Learning Community at Scalecited by 8
- 2023Shaping New Norms for AIcited by 7
- 2026Let There Be Claws: An Early Social Network Analysis of AI Agents on Moltbookcited by 7
- 2025Emergent Coordinated Behaviors in Networked LLM Agents: Modeling the Strategic Dynamics of Information Operationscited by 6
- 2025Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystemcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Peer Influence across Heterogeneous AI Modelscited by 0
- 2026Peer Influence across Heterogeneous AI Models2 Oct 2026
Similar, not cited
Close in wording, no citation either way
- 2026Conformity Generates Collective Misalignment in AI Agents Societies
- 2025Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails
- 2026Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff
- 2026Increasing intelligence in AI agents can worsen collective outcomes
- 2026Agentic Microphysics: A Manifesto for Generative AI Safety
- 2026Agentic Pressure: The Endogenous Entropy of Reliable Autonomy
- 2026Conformity and Social Impact on AI Agents
- 2026Formal Mechanisms for Market Stability in Self-Interested Agent Societies: A Marketplace Simulation Study