mlsanityData to 7 Oct 2026

Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications

arXiv 2303.08060 ↗First posted 14 Mar 2023cs.AI cs.NEcited by 16

Not on the map. Why?

Abstract (arXiv)

The advent of AlphaGo and its successors marked the beginning of a new paradigm in playing games using artificial intelligence. This was achieved by combining Monte Carlo tree search, a planning procedure, and deep learning. While the impact on the domain of games has been undeniable, it is less clear how useful similar approaches are in applications beyond games and how they need to be adapted from the original methodology. We review 129 peer-reviewed articles detailing the application of neural Monte Carlo tree search methods in domains other than games. Our goal is to systematically assess how such methods are structured in practice and if their success can be extended to other domains. We find applications in a variety of domains, many distinct ways of guiding the tree search using learned policy and value functions, and various training methods. Our review maps the current landscape of algorithms in the family of neural monte carlo tree search as they are applied to practical problems, which is a first step towards a more principled way of designing such algorithms for specific problems and their requirements.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 40 of its 168 references that are in the map. First posted on arXiv 14 Mar 2023.

Builds on

40 of 168 references.

References · 40

  1. 2016OpenAI Gymcited by 1,684
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  3. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 245
  4. 2017Learning to Plan Chemical Synthesescited by 175
  5. 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 158
  6. 2020Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learningcited by 139
  7. 2019Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Drivingcited by 52
  8. 2021Learning to design drug-like molecules in three-dimensional space using deep generative modelscited by 37
  9. 2018M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Searchcited by 34
  10. 2017ChemTS: An Efficient Python Library for de novo Molecular Generationcited by 32
  11. 2022MuZero with Self-competition for Rate Control in VP9 Video Compressioncited by 27
  12. 2018Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimizationcited by 25
28 more
  1. 2017Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environmentscited by 19
  2. 2018Meta-modeling game for deriving theoretical-consistent, micro-structural-based traction-separation laws via deep reinforcement learningcited by 19
  3. 2018A0C: Alpha Zero in Continuous Action Spacecited by 17
  4. 2021Adapting User Interfaces with Model-based Reinforcement Learningcited by 16
  5. 2019Coloring Big Graphs with AlphaGoZerocited by 13
  6. 2021SOLO: Search Online, Learn Offline for Combinatorial Optimization Problemscited by 9
  7. 2021Qubit Routing using Graph Neural Network aided Monte Carlo Tree Searchcited by 9
  8. 2019Deep Reinforcement Learning for Synthesizing Functions in Higher-Order Logiccited by 8
  9. 2020Online Scheduling of a Residential Microgrid via Monte-Carlo Tree Search and a Learned Modelcited by 8
  10. 2021Self-play Learning Strategies for Resource Assignment in Open-RAN Networkscited by 8
  11. 2021The Role of Entropy in Guiding a Connection Provercited by 8
  12. 2020Prolog Technology Reinforcement Learning Provercited by 7
  13. 2020Solve Traveling Salesman Problem by Monte Carlo Tree Search and Deep Neural Networkcited by 7
  14. 2019Learning Self-Game-Play Agents for Combinatorial Optimization Problemscited by 6
  15. 2017Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policiescited by 5
  16. 2021Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environmentscited by 5
  17. 2022Learning to design without prior data: Discovering generalizable design strategies using deep learning and tree searchcited by 5
  18. 2020A Unifying Framework for Reinforcement Learning and Planningcited by 4
  19. 2021Costly Features Classification using Monte Carlo Tree Searchcited by 4
  20. 2019Deep Reinforcement Learning-Based Topology Optimization for Self-Organized Wireless Sensor Networkscited by 3
  21. 2019Self-Play Learning Without a Reward Metriccited by 3
  22. 2020Mastering the working sequence in human-robot collaborative assembly based on reinforcement learningcited by 3
  23. 2021Learning Robust Scheduling with Search and Attentioncited by 3
  24. 2018AlphaSeq: Sequence Discovery with Deep Reinforcement Learningcited by 2
  25. 2020A reinforcement learning application of guided Monte Carlo Tree Search algorithm for beam orientation selection in radiation therapycited by 2
  26. 2021Deep Reinforcement Learning for Dynamic Spectrum Sharing of LTE and NRcited by 2
  27. 2022Investigation of reinforcement learning for shape optimization of profile extrusion diescited by 2
  28. 2022Karolos: An Open-Source Reinforcement Learning Framework for Robot-Task Environmentscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 9; 2026: 5.

16 papers.

Sort:

By area:

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  2. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  3. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  4. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  5. 2025DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Searchcited by 11
  6. 2026Planning Task Shielding: Detecting and Repairing Flaws in Planning Tasks through Turning them Unsolvablecited by 1
  7. 2024MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddingscited by 0
  8. 2025Adaptive Bias Generalized Rollout Policy Adaptation on the Flexible Job-Shop Scheduling Problemcited by 0
  9. 2025DyRo-MCTS: A Robust Monte Carlo Tree Search Approach to Dynamic Job Shop Schedulingcited by 0
  10. 2025Unveiling Interesting Insights: Monte Carlo Tree Search for Knowledge Discoverycited by 0
  11. 2025Optimizing Fairness in Production Planning: A Human-Centric Approach to Machine and Workforce Allocationcited by 0
  12. 2025Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Searchcited by 0
4 more
  1. 2026Monte-Carlo Tree Search with Neural Network Guidance for Lane-Free Autonomous Drivingcited by 0
  2. 2026Optimizing Neural Network Architecture for Medical Image Segmentation Using Monte Carlo Tree Searchcited by 0
  3. 2026AlphaTransit: Learning to Design City-scale Transit Routescited by 0
  4. 2026StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesiscited by 0