Memory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Games
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
This paper proposes Memory-Augmented State Machine Prompting (MASMP), a novel framework for LLM agents in real-time strategy games. Addressing key challenges like hallucinations and fragmented decision-making in existing approaches, MASMP integrates state machine prompting with memory mechanisms to unify structured actions with long-term tactical coherence. The framework features: (1) a natural language-driven state machine architecture that guides LLMs to emulate finite state machines and behavior trees through prompts, and (2) a lightweight memory module preserving strategic variables (e.g., tactics, priority units) across decision cycles. Experiments in StarCraft II demonstrate MASMP's 60% win rate against the hardest built-in AI (Lv7), vastly outperforming baselines (0%). Case studies reveal the method retains LLMs' semantic comprehension while resolving the "Knowing-Doing Gap" through strict state-action mapping, achieving both interpretability and FSM-like reliability. This work establishes a new paradigm for combining neural and symbolic AI in complex decision-making.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 8 of its 13 references that are in the map. First posted on arXiv 21 Oct 2025.
Builds on
8 of 13 references.
Closest ancestors · 4
Same direction, newest first
- 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities22 Apr
- 2024SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models31 Jan
- 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach19 Dec
- 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory25 May
Other references · 4
- 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
- 2018TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Gamecited by 14
- 2024COA-GPT: Generative Pre-trained Transformers for Accelerated Course of Action Development in Military Operationscited by 12
- 2024LLM-PySC2: Starcraft II learning environment for Large Language Modelscited by 9
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026Evaluating Memory Structure in LLM Agentscited by 12
- 2026World Models for Policy Refinement in StarCraft IIcited by 0
- 2026World Models for Policy Refinement in StarCraft II16 Feb 2026
- 2026Evaluating Memory Structure in LLM Agents11 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
- 2026AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents
- 2026LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning
- 2024STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making
- 2026Towards Improving Sequential Decision-Making in LLM Agents via Experience Memory
- 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
- 2026Sparks of Cooperative Reasoning: LLMs as Strategic Hanabi Agents
- 2026BotzoneBench: Scalable LLM Evaluation via Graded AI Anchors