# Aligned Alone, Misaligned Together: Forecasting Adversarial Capture in LLM Agent Populations

arXiv: https://arxiv.org/abs/2608.22444 · HTML: https://mlsanity.com/paper/2608.22444/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 9 of 9)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Open Problems in Cooperative AI](https://mlsanity.com/paper/2012.08630/) (2020)
- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [AI Research Considerations for Human Existential Safety (ARCHES)](https://mlsanity.com/paper/2006.04948/) (2020)
- [The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process](https://mlsanity.com/paper/2506.01080/) (2025)
- [MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework](https://mlsanity.com/paper/2504.21582/) (2025)
- [Opinion control in complex networks](https://mlsanity.com/paper/1412.2170/) (2014)
- [Steering the distribution of agents in mean-field and cooperative games](https://mlsanity.com/paper/1712.03578/) (2017)
- [When Is Emergent Consensus Real? A Measured Coupling Gain and a Validity Diagnostic for LLM Agent Societies](https://mlsanity.com/paper/2606.22203/) (2026)

## Built on by (most cited first, 2 of 2)

- [LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems](https://mlsanity.com/paper/2609.33495/) (2026)
- [Mean field games as a tool for AI safety: a worked example from the July 2026 Hugging Face incident](https://mlsanity.com/paper/2610.00902/) (2026)

## Abstract (arXiv)

The unit of AI safety evaluation is still the individual model, yet language-model agents are increasingly deployed in interacting populations that read and write one another's decisions. This raises a question no single-agent audit can answer: an agent that is well-calibrated on its own may still be pulled toward a different decision by the agents around it. We study this on a security-triage task, where populations of language-model monitors decide whether to escalate or dismiss alerts, and into which we can inject a committed minority that always pushes one way. We find that two alerts a single agent judges almost identically on its own can drive collective behavior far apart, so auditing any one member need not reveal what the population will do. Yet that collective behavior can be predicted in advance. From a population's benign, adversary-free operation alone, we calibrate a response function that forecasts, before any attack is run, how far a committed minority will later move it. We then ask what shifts the outcome and find that letting agents see each other's reasoning neutralizes a weak attack, while only delaying it against a strong one, turning the question from whether the population converges on the adversaries' choice into when. Finally, we exclude the hypothesis of capture being an irreversible trap: once the committed agents are removed, the population drifts back toward where it began, so capture is a temporary state. Alignment in isolation is not alignment in a population, yet what a population will do under attack can be read in advance, from how it behaves before any adversary arrives.