mlsanityData to 7 Oct 2026

Escalation Risks from Language Models in Military and Diplomatic Decision-Making

arXiv 2401.03408 ↗First posted 7 Jan 2024cs.AI cs.CL cs.CYcited by 61

Not on the map. Why?

Abstract (arXiv)

Governments are increasingly considering integrating autonomous AI agents in high-stakes military and foreign-policy decision-making, especially with the emergence of advanced generative AI models like GPT-4. Our work aims to scrutinize the behavior of multiple AI agents in simulated wargames, specifically focusing on their predilection to take escalatory actions that may exacerbate multilateral conflicts. Drawing on political science and international relations literature about escalation dynamics, we design a novel wargame simulation and scoring framework to assess the escalation risks of actions taken by these agents in different scenarios. Contrary to prior studies, our research provides both qualitative and quantitative insights and focuses on large language models (LLMs). We find that all five studied off-the-shelf LLMs show forms of escalation and difficult-to-predict escalation patterns. We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons. Qualitatively, we also collect the models' reported reasonings for chosen actions and observe worrying justifications based on deterrence and first-strike tactics. Given the high stakes of military and foreign-policy contexts, we recommend further examination and cautious consideration before deploying autonomous language model agents for strategic military or diplomatic decision-making.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 1 of its 27 references that are in the map. First posted on arXiv 7 Jan 2024.

Builds on

1 of 27 references.

References · 1

  1. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 21; 2026: 24.

61 papers.

Sort:

By area:

  1. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 112
  2. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  3. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  4. 2024Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementationcited by 27
  5. 2024(Ir)rationality and Cognitive Biases in Large Language Modelscited by 23
  6. 2024Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulationscited by 20
  7. 2024On Large Language Models in National Security Applicationscited by 18
  8. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  9. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 17
  10. 2024The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?cited by 14
  11. 2024AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Researchcited by 13
  12. 2024Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilitiescited by 12
38 more
  1. 2024Measuring Free-Form Decision-Making Inconsistency of Language Models in Military Crisis Simulationscited by 11
  2. 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 11
  3. 2024Societal Adaptation to Advanced AIcited by 10
  4. 2024OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safetycited by 9
  5. 2024Open-Ended Wargames with Large Language Modelscited by 8
  6. 2025A Survey of Theory of Mind in Large Language Models: Evaluations, Representations, and Safety Riskscited by 8
  7. 2025The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systemscited by 8
  8. 2025Critical Foreign Policy Decisions (CFPD)-Benchmark: Measuring Diplomatic Preferences in Large Language Modelscited by 7
  9. 2026AI Arms and Influence: Frontier Models Exhibit Sophisticated Reasoning in Simulated Nuclear Crisescited by 7
  10. 2025Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibilitycited by 6
  11. 2024Large Language Models in Politics and Democracy: A Comprehensive Surveycited by 5
  12. 2024Markovian Transformers for Informative Language Modelingcited by 4
  13. 2025The Prompt War: How AI Decides on a Military Interventioncited by 4
  14. 2026Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communitiescited by 4
  15. 2025Military AI Needs Technically-Informed Regulation to Safeguard AI Research and its Applicationscited by 3
  16. 2025Integrators at War: Mediating in AI-assisted Resort-to-Force Decisionscited by 2
  17. 2025Identifying Cooperative Personalities in Multi-agent Contexts through Personality Steering with Representation Engineeringcited by 2
  18. 2025Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.iicited by 2
  19. 2025Abstract Counterfactuals for Language Model Agentscited by 2
  20. 2025Military AI Cyber Agents (MAICAs) Constitute a Global Threat to Critical Infrastructurecited by 2
  21. 2025Against racing to AGI: Cooperation, deterrence, and catastrophic riskscited by 2
  22. 2025Managing Escalation in Off-the-Shelf Large Language Modelscited by 2
  23. 2026A Scalable Entity-Based Framework for Auditing Bias in Large Language Modelscited by 2
  24. 2026CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization Vcited by 2
  25. 2025Social preferences with unstable interactive reasoning: Large language models in economic trust gamescited by 1
  26. 2025The Problem of Algorithmic Collisions: Mitigating Unforeseen Risks in a Connected Worldcited by 1
  27. 2025Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanationscited by 1
  28. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 1
  29. 2025A Criminology of Machinescited by 1
  30. 2026Effects of personality steering on cooperative behavior in Large Language Model agentscited by 1
  31. 2026Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generationcited by 1
  32. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  33. 2025Governing Automated Strategic Intelligencecited by 0
  34. 2026Ignore All Previous Instructions: Jailbreaking as a de-escalatory peace building practise to resist LLM social media botscited by 0
  35. 2026Implementing surrogate goals for safer bargaining in LLM-based agentscited by 0
  36. 2026The Weaponization of Computer Vision: Tracing Military-Surveillance Ties through Conference Sponsorshipcited by 0
  37. 2026Multi-Agent Strategic Games with LLMscited by 0
  38. 2026Human Decision-Making with Persuasive and Narrative LLM Explanationscited by 0

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026LLMs as Strategic Actors: Behavioral Alignment, Risk Calibration, and Argumentation Framing in Geopolitical Simulations
  2. 2026When AI Navigates the Fog of War
  3. 2025DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
  4. 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars
  5. 2025Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy
  6. 2023Strategic Reasoning with Language Models
  7. 2025Large Language Models as Pok'emon Battle Agents: Strategic Play and Content Generation
  8. 2026To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulation