mlsanityData to 7 Oct 2026

Look-ahead Reasoning with a Learned Model in Imperfect Information Games

arXiv 2510.05048 ↗First posted 6 Oct 2025cs.AI cs.GTcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Test-time reasoning significantly enhances pre-trained AI agents' performance. However, it requires an explicit environment model, often unavailable or overly complex in real-world scenarios. While MuZero enables effective model learning for search in perfect information games, extending this paradigm to imperfect information games presents substantial challenges due to more nuanced look-ahead reasoning techniques and large number of states relevant for individual decisions. This paper introduces an algorithm LAMIR that learns an abstracted model of an imperfect information game directly from the agent-environment interaction. During test time, this trained model is used to perform look-ahead reasoning. The learned abstraction limits the size of each subgame to a manageable size, making theoretically principled look-ahead reasoning tractable even in games where previous methods could not scale. We empirically demonstrate that with sufficient capacity, LAMIR learns the exact underlying game structure, and with limited capacity, it still learns a valuable abstraction, which improves game playing performance of the pre-trained agents even in large games.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 29 of its 55 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

29 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Mastering Board Games by External and Internal Planning with Language Models2 Dec

Other references · 28

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2020Mastering Atari with Discrete World Modelscited by 990
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  6. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  7. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  8. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  9. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  10. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 110
  11. 2018Deep Counterfactual Regret Minimizationcited by 102
  12. 2014Solving Large Imperfect Information Games Using CFR+cited by 78
16 more
  1. 2020From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularizationcited by 59
  2. 2020DREAM: Deep Regret minimization with Advantage baselines and Model-free learningcited by 36
  3. 2019Rethinking Formal Models of Partially Observable Multiagent Decision Makingcited by 35
  4. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  5. 2013Solving Imperfect Information Games Using Decompositioncited by 27
  6. 2018Depth-Limited Solving for Imperfect-Information Gamescited by 24
  7. 2018Variance Reduction in Monte Carlo Counterfactual Regret Minimization (VR-MCCFR) for Extensive Form Games using Baselinescited by 24
  8. 2021Student of Games: A unified learning algorithm for both perfect and imperfect information gamescited by 22
  9. 2021Subgame solving without common knowledgecited by 11
  10. 2024RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learningcited by 10
  11. 2023Abstracting Imperfect Information Away from Two-Player Zero-Sum Gamescited by 6
  12. 2025General search techniques without common knowledge for imperfect-information games, and application to superhuman Fog of War chesscited by 5
  13. 2018Automated Construction of Bounded-Loss Imperfect-Recall Abstractions in Extensive-Form Gamescited by 2
  14. 2025Adapting Beyond the Depth Limit: Counter Strategies in Large Imperfect Information Gamescited by 2
  15. 2023Look-ahead Search on Top of Policy Networks in Imperfect Information Gamescited by 1
  16. 2025On the Power of Perturbation under Sampling in Solving Extensive-Form Gamescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4
  2. 2026SAGE: Structured Strategic Reasoning for Efficient LLM Game Playing
  3. 2026BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search
  4. 2026AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games
  5. 2025Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games
  6. 2026ProAct: Agentic Lookahead in Interactive Environments
  7. 2023SPRING: Studying the Paper and Reasoning to Play Games
  8. 2026Abstraction Agent