mlsanityData to 7 Oct 2026

What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinking

arXiv 2509.04791 ↗First posted 5 Sep 2025cs.AIcited by 3

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

LLMs struggle with decision-making in high-stakes environments like MOBA games, primarily due to a lack of proactive reasoning and limited understanding of complex game dynamics. To address this, we propose What-if Analysis LLM (WiA-LLM), a framework that trains an LLM as an explicit, language-based world model. Instead of representing the environment in latent vectors, WiA-LLM uses natural language to simulate how the game state evolves over time in response to candidate actions, and provides textual justifications for these predicted outcomes. WiA-LLM is trained in two stages: supervised fine-tuning on human-like reasoning traces, followed by reinforcement learning with outcome-based rewards based on the alignment between predicted and actual future states. In the Honor of Kings (HoK) environment, WiA-LLM attains 74.2% accuracy (27%uparrow vs. base model) in forecasting game-state changes. In addition, WiA-LLM demonstrate strategic behavior more closely aligned with expert players than purely reactive LLMs, indicating enhanced foresight and expert-like decision-making.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 30 of its 43 references that are in the map. First posted on arXiv 5 Sep 2025.

Builds on

30 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models29 Aug
  2. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction11 Feb

Other references · 28

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  9. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  11. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  12. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
16 more
  1. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  2. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  3. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  4. 2024CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluationcited by 139
  5. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  6. 2020Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problemscited by 60
  7. 2024Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilitiescited by 48
  8. 2025Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Modelscited by 33
  9. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 27
  10. 2024PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language Modelscited by 25
  11. 2023Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHFcited by 20
  12. 2024CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLMcited by 18
  13. 2025Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidancecited by 14
  14. 2024Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarkscited by 10
  15. 2022Time Series Forecasting (TSF) Using Various Deep Learning Modelscited by 4
  16. 2021Augmenting Decision Making via Interactive What-If Analysiscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steeringcited by 4
  2. 2026Composable Crystals: Controllable Materials Discovery via Concept Learningcited by 1
  3. 2026World Agent: Can Language Models Keep a World Running?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Tracing LLM Reasoning Processes with Strategic Games: A Framework for Planning, Revision, and Resource-Constrained Decision Making
  2. 2025WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-making
  3. 2024Large Language Models as Agents in Two-Player Games
  4. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
  5. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
  6. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  7. 2026BotzoneBench: Scalable LLM Evaluation via Graded AI Anchors
  8. 2025Observer, Not Player: Simulating Theory of Mind in LLMs through Game Observation