# Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications

arXiv: https://arxiv.org/abs/2303.08060 · HTML: https://mlsanity.com/paper/2303.08060/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 168 references that are in the map.
- First posted on arXiv 14 Mar 2023.

## Builds on (most cited first, 30 of 40)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learning](https://mlsanity.com/paper/2010.12367/) (2020)
- [Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving](https://mlsanity.com/paper/1905.02680/) (2019)
- [Learning to design drug-like molecules in three-dimensional space using deep generative models](https://mlsanity.com/paper/2104.08474/) (2021)
- [M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search](https://mlsanity.com/paper/1802.04394/) (2018)
- [ChemTS: An Efficient Python Library for de novo Molecular Generation](https://mlsanity.com/paper/1710.00616/) (2017)
- [MuZero with Self-competition for Rate Control in VP9 Video Compression](https://mlsanity.com/paper/2202.06626/) (2022)
- [Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization](https://mlsanity.com/paper/1807.01672/) (2018)
- [Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments](https://mlsanity.com/paper/1703.07887/) (2017)
- [Meta-modeling game for deriving theoretical-consistent, micro-structural-based traction-separation laws via deep reinforcement learning](https://mlsanity.com/paper/1810.10535/) (2018)
- [A0C: Alpha Zero in Continuous Action Space](https://mlsanity.com/paper/1805.09613/) (2018)
- [Adapting User Interfaces with Model-based Reinforcement Learning](https://mlsanity.com/paper/2103.06807/) (2021)
- [Coloring Big Graphs with AlphaGoZero](https://mlsanity.com/paper/1902.10162/) (2019)
- [SOLO: Search Online, Learn Offline for Combinatorial Optimization Problems](https://mlsanity.com/paper/2104.01646/) (2021)
- [Qubit Routing using Graph Neural Network aided Monte Carlo Tree Search](https://mlsanity.com/paper/2104.01992/) (2021)
- [Deep Reinforcement Learning for Synthesizing Functions in Higher-Order Logic](https://mlsanity.com/paper/1910.11797/) (2019)
- [Online Scheduling of a Residential Microgrid via Monte-Carlo Tree Search and a Learned Model](https://mlsanity.com/paper/2005.06161/) (2020)
- [Self-play Learning Strategies for Resource Assignment in Open-RAN Networks](https://mlsanity.com/paper/2103.02649/) (2021)
- [The Role of Entropy in Guiding a Connection Prover](https://mlsanity.com/paper/2105.14706/) (2021)
- [Learning Self-Game-Play Agents for Combinatorial Optimization Problems](https://mlsanity.com/paper/1903.03674/) (2019)
- [Prolog Technology Reinforcement Learning Prover](https://mlsanity.com/paper/2004.06997/) (2020)
- [Solve Traveling Salesman Problem by Monte Carlo Tree Search and Deep Neural Network](https://mlsanity.com/paper/2005.06879/) (2020)
- [Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments](https://mlsanity.com/paper/2104.09705/) (2021)
- [Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies](https://mlsanity.com/paper/1702.00020/) (2017)
- [Learning to design without prior data: Discovering generalizable design strategies using deep learning and tree search](https://mlsanity.com/paper/2211.15068/) (2022)
- [A Unifying Framework for Reinforcement Learning and Planning](https://mlsanity.com/paper/2006.15009/) (2020)

## Built on by (most cited first, 16 of 16)

- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [Top-Down Semantic Refinement for Image Captioning](https://mlsanity.com/paper/2510.22391/) (2025)
- [Diversifying AI: Towards Creative Chess with AlphaZero](https://mlsanity.com/paper/2308.09175/) (2023)
- [DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search](https://mlsanity.com/paper/2509.25454/) (2025)
- [Planning Task Shielding: Detecting and Repairing Flaws in Planning Tasks through Turning them Unsolvable](https://mlsanity.com/paper/2604.07042/) (2026)
- [MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddings](https://mlsanity.com/paper/2411.00287/) (2024)
- [Adaptive Bias Generalized Rollout Policy Adaptation on the Flexible Job-Shop Scheduling Problem](https://mlsanity.com/paper/2505.08451/) (2025)
- [DyRo-MCTS: A Robust Monte Carlo Tree Search Approach to Dynamic Job Shop Scheduling](https://mlsanity.com/paper/2509.21902/) (2025)
- [Unveiling Interesting Insights: Monte Carlo Tree Search for Knowledge Discovery](https://mlsanity.com/paper/2510.00876/) (2025)
- [Optimizing Fairness in Production Planning: A Human-Centric Approach to Machine and Workforce Allocation](https://mlsanity.com/paper/2510.01094/) (2025)
- [Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Search](https://mlsanity.com/paper/2512.21648/) (2025)
- [Monte-Carlo Tree Search with Neural Network Guidance for Lane-Free Autonomous Driving](https://mlsanity.com/paper/2601.09353/) (2026)
- [Optimizing Neural Network Architecture for Medical Image Segmentation Using Monte Carlo Tree Search](https://mlsanity.com/paper/2602.22361/) (2026)
- [AlphaTransit: Learning to Design City-scale Transit Routes](https://mlsanity.com/paper/2605.28730/) (2026)
- [StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesis](https://mlsanity.com/paper/2606.04246/) (2026)

## Abstract (arXiv)

The advent of AlphaGo and its successors marked the beginning of a new paradigm in playing games using artificial intelligence. This was achieved by combining Monte Carlo tree search, a planning procedure, and deep learning. While the impact on the domain of games has been undeniable, it is less clear how useful similar approaches are in applications beyond games and how they need to be adapted from the original methodology. We review 129 peer-reviewed articles detailing the application of neural Monte Carlo tree search methods in domains other than games. Our goal is to systematically assess how such methods are structured in practice and if their success can be extended to other domains. We find applications in a variety of domains, many distinct ways of guiding the tree search using learned policy and value functions, and various training methods. Our review maps the current landscape of algorithms in the family of neural monte carlo tree search as they are applied to practical problems, which is a first step towards a more principled way of designing such algorithms for specific problems and their requirements.