# Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search

arXiv: https://arxiv.org/abs/2012.07910 · HTML: https://mlsanity.com/paper/2012.07910/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 5 of 5)

- [Are AlphaZero-like Agents Robust to Adversarial Perturbations?](https://mlsanity.com/paper/2211.03769/) (2022)
- [Spending Thinking Time Wisely: Accelerating MCTS with Virtual Expansions](https://mlsanity.com/paper/2210.12628/) (2022)
- [Amplifying Exploration in Monte-Carlo Tree Search by Focusing on the Unknown](https://mlsanity.com/paper/2402.08511/) (2024)
- [A Bayesian Approach to Online Planning](https://mlsanity.com/paper/2406.02103/) (2024)
- [Engineering Efficient Self-Play Chess: Search, Replay, and Throughput Under Limited Compute](https://mlsanity.com/paper/2609.37447/) (2026)

## Abstract (arXiv)

Monte Carlo tree search (MCTS) has achieved state-of-the-art results in many domains such as Go and Atari games when combining with deep neural networks (DNNs). When more simulations are executed, MCTS can achieve higher performance but also requires enormous amounts of CPU and GPU resources. However, not all states require a long searching time to identify the best action that the agent can find. For example, in 19x19 Go and NoGo, we found that for more than half of the states, the best action predicted by DNN remains unchanged even after searching 2 minutes. This implies that a significant amount of resources can be saved if we are able to stop the searching earlier when we are confident with the current searching result. In this paper, we propose to achieve this goal by predicting the uncertainty of the current searching status and use the result to decide whether we should stop searching. With our algorithm, called Dynamic Simulation MCTS (DS-MCTS), we can speed up a NoGo agent trained by AlphaZero 2.5 times faster while maintaining a similar winning rate. Also, under the same average simulation count, our method can achieve a 61% winning rate against the original program.