mlsanityData to 14 Oct 2026

Monte Carlo Tree Search: A Review of Recent Modifications and Applications

arXiv 2103.04931 ↗First posted 8 Mar 2021cs.AI cs.LG cs.MAcited by 161

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Monte Carlo Tree Search (MCTS) is a powerful approach to designing game-playing bots or solving sequential decision problems. The method relies on intelligent tree search that balances exploration and exploitation. MCTS performs random sampling in the form of simulations and stores statistics of actions to make more educated choices in each subsequent iteration. The method has become a state-of-the-art technique for combinatorial games, however, in more complex games (e.g. those with high branching factor or real-time ones), as well as in various practical domains (e.g. transportation, scheduling or security) an efficient MCTS application often requires its problem-dependent modification or integration with other techniques. Such domain-specific modifications and hybrid approaches are the main focus of this survey. The last major MCTS survey has been published in 2012. Contributions that appeared since its release are of particular interest for this review.

Key facts. Cited by 161 papers in the map (data through 14 Oct 2026). Builds on 24 of its 255 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

24 of 255 references.

References · 24

  1. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
  2. 2017Learning to Plan Chemical Synthesescited by 179
  3. 2018Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithmscited by 13
  4. 2014Monte Carlo Tree Search with Heuristic Evaluations using Implicit Minimax Backupscited by 11
  5. 2019Robust and Adaptive Planning under Model Uncertaintycited by 11
  6. 2018Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Searchcited by 9
  7. 2020Rolling Horizon Evolutionary Algorithms for General Video Game Playingcited by 9
  8. 2019Action Guidance with MCTS for Deep Reinforcement Learningcited by 8
  9. 2020Convex Hull Monte-Carlo Tree Searchcited by 8
  10. 2019Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimatescited by 6
  11. 2017Multi-Labelled Value Networks for Computer Gocited by 5
  12. 2017Combining Strategic Learning and Tactical Search in Real-Time Strategy Gamescited by 4
12 more
  1. 2015Scaling Monte Carlo Tree Search on Intel Xeon Phicited by 3
  2. 2015Combining Monte-Carlo and Hyper-heuristic methods for the Multi-mode Resource-constrained Multi-project Scheduling Problemcited by 3
  3. 2017Analysis of Vanilla Rolling Horizon Evolution Parameters in General Video Game Playingcited by 3
  4. 2019Double-oracle sampling method for Stackelberg Equilibrium approximation in general-sum extensive-form gamescited by 3
  5. 2020Does it matter how well I know what you're thinking? Opponent Modelling in an RTS gamecited by 3
  6. 2018Monte Carlo Methods for the Game Kingdominocited by 2
  7. 2019Monte-Carlo Tree Search for Simulation-based Strategy Analysiscited by 2
  8. 2020Integrating Acting, Planning and Learning in Hierarchical Operational Modelscited by 2
  9. 2014MTD(f), A Minimax Algorithm Faster Than NegaScoutcited by 1
  10. 2018Traditional Wisdom and Monte Carlo Tree Search Face-to-Face in the Card Game Scoponecited by 1
  11. 2018Monte Carlo Tree Search with Scalable Simulation Periods for Continuously Running Taskscited by 1
  12. 2019Anchoring Theory in Sequential Stackelberg Gamescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 9; 2023: 19; 2024: 39; 2025: 56; 2026: 32.

161 papers.

Sort:

By area:

  1. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  2. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  3. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  4. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  5. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  6. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
  7. 2024ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learningcited by 62
  8. 2025TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modelingcited by 61
  9. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 54
  10. 2025KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systemscited by 37
  11. 2024Efficient Reinforcement Learning with Large Language Model Priorscited by 30
  12. 2025AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verificationcited by 23
38 more
  1. 2024Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlocited by 22
  2. 2024Procedural Content Generation in Games: A Survey with Insights on Emerging LLM Integrationcited by 22
  3. 2025KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Searchcited by 22
  4. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  5. 2024Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generationcited by 21
  6. 2025TripCraft: A Benchmark for Spatio-Temporally Fine Grained Travel Planningcited by 21
  7. 2025HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial Optimization Challengescited by 21
  8. 2025WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesiscited by 21
  9. 2023No Train Still Gain. Unleash Mathematical Reasoning of Large Language Models with Monte Carlo Tree Search Guided by Energy Functioncited by 20
  10. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  11. 2025Failure-Driven Workflow Refinementcited by 20
  12. 2025SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQLcited by 19
  13. 2023Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applicationscited by 16
  14. 2024Enhancing Mathematical Reasoning in LLMs by Stepwise Correctioncited by 16
  15. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  16. 2025Tool-integrated Reinforcement Learning for Repo Deep Searchcited by 15
  17. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  18. 2024QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCEcited by 14
  19. 2025Improving Pre-Trained Vision-Language-Action Policies with Model-Based Searchcited by 13
  20. 2025VLA-Reasoner: Empowering Vision-Language-Action Models with Reasoning via Online Monte Carlo Tree Searchcited by 13
  21. 2023OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workloadcited by 12
  22. 2024UniZero: Generalized and Efficient Planning with Scalable Latent World Modelscited by 12
  23. 2025From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuningcited by 12
  24. 20253DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scalecited by 12
  25. 2025BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVscited by 10
  26. 2025Generalizability of Large Language Model-Based Agents: A Comprehensive Surveycited by 10
  27. 2025Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMscited by 10
  28. 2024Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designscited by 9
  29. 2024Reinforcement learning-based architecture search for quantum machine learningcited by 9
  30. 2024Adaptive Draft-Verification for Efficient Large Language Model Decodingcited by 9
  31. 2023Finding Increasingly Large Extremal Graphs with AlphaZero and Tabu Searchcited by 8
  32. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  33. 2025TripTide: A Benchmark for Adaptive Travel Planning under Disruptionscited by 8
  34. 2022Epistemic Monte Carlo Tree Searchcited by 7
  35. 2023Adaptive Anytime Multi-Agent Path Finding Using Bandit-Based Large Neighborhood Searchcited by 7
  36. 2024Towards Measuring Goal-Directedness in AI Systemscited by 7
  37. 2024MC-NEST: Enhancing Mathematical Reasoning in Large Language Models leveraging a Monte Carlo Self-Refine Treecited by 7
  38. 2023C-MCTS: Safe Planning with Monte Carlo Tree Searchcited by 6

50 most cited of 161 All 161 (JSON)