Shall We Play a Game? Language Models for Open-ended Wargames
Not on the map. Why?
Abstract (arXiv)
LLM-based social simulations can make a generated transcript look like a single behavioral signal, but the model behind that transcript may be doing several different jobs: choosing what an actor says or does, deciding what happens after an action, or both. The difference matters especially in open-ended wargames, where models are prized for handling unusual actions and ambiguous consequences. We report a scoping review of 223 de-duplicated AI-in-wargames and strategic-simulation papers retrieved through May 1, 2026, describing each simulation by its model-control profile: whether the language model has open-ended control over player actions, adjudication, or both. Only 20 of 223 studies (~9%) give language models both roles. Before treating LM outputs as social simulations, researchers need to know how much creative control the model has over actions and consequences. For open-ended simulations, fidelity depends not only on whether agents behave plausibly, but also on whether language models can reliably act as adjudicators or world models.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 155 of its 312 references that are in the map. First posted on arXiv 21 Sep 2025.
Builds on
155 of 312 references.
References · 155
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
- 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
38 more
- 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
- 2023Playing repeated games with Large Language Modelscited by 217
- 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
- 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
- 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
- 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Warscited by 145
- 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
- 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
- 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 126
- 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 126
- 2024Emergent social conventions and collective bias in LLM populationscited by 123
- 2024A Survey on Large Language Model-Based Game Agentscited by 112
- 2024Systematic Biases in LLM Simulations of Debatescited by 99
- 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
- 2024On scalable oversight with weak LLMs judging strong LLMscited by 99
- 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 96
- 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 95
- 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 95
- 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
- 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 86
- 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
- 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approachcited by 79
- 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
- 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 75
- 2024OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Codecited by 74
- 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 70
- 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 68
- 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 67
- 2023Strategic Reasoning with Language Modelscited by 64
- 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 63
- 2024Escalation Risks from Language Models in Military and Diplomatic Decision-Makingcited by 63
- 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 56
- 2023Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agentscited by 54
- 2023Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framingcited by 51
- 2024Project Sid: Many-agent simulations toward AI civilizationcited by 49
- 2022Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planningcited by 48
- 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
- 2023Welfare Diplomacy: Benchmarking Language Model Cooperationcited by 45
50 most cited of 155 All 155 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Playing games with Large language models: Randomness and strategy
- 2025Collaboration and Conflict between Humans and Language Models through the Lens of Game Theory
- 2024clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents
- 2024Enhancing Agent Learning through World Dynamics Modeling
- 2024Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games
- 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation
- 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
- 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs