# Adaptive Warm-Start MCTS in AlphaZero-like Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2105.06136 · HTML: https://mlsanity.com/paper/2105.06136/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 7 of 7)

- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Accelerating Self-Play Learning in Go](https://mlsanity.com/paper/1902.10565/) (2019)
- [Tackling Morpion Solitaire with AlphaZero-likeRanked Reward Reinforcement Learning](https://mlsanity.com/paper/2006.07970/) (2020)
- [Assessing the Potential of Classical Q-learning in General Game Playing](https://mlsanity.com/paper/1810.06078/) (2018)
- [Warm-Start AlphaZero Self-Play Search Enhancements](https://mlsanity.com/paper/2004.12357/) (2020)
- [Analysis of Hyper-Parameters for Small Games: Iterations or Epochs in Self-Play?](https://mlsanity.com/paper/2003.05988/) (2020)

## Built on by (most cited first, 6 of 6)

- [AlphaDDA: Strategies for Adjusting the Playing Strength of a Fully Trained AlphaZero System to a Suitable Human Training Partner](https://mlsanity.com/paper/2111.06266/) (2021)
- [Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Four](https://mlsanity.com/paper/2202.11797/) (2022)
- [Generalised agent for solving higher board states of tic tac toe using Reinforcement Learning](https://mlsanity.com/paper/2212.12252/) (2022)
- [Towards Tackling MaxSAT by Combining Nested Monte Carlo with Local Search](https://mlsanity.com/paper/2302.13225/) (2023)
- [AlphaViT: A flexible game-playing AI for multiple games and variable board sizes](https://mlsanity.com/paper/2408.13871/) (2024)
- [TSS GAZ PTP: Towards Improving Gumbel AlphaZero with Two-stage Self-play for Multi-constrained Electric Vehicle Routing Problems](https://mlsanity.com/paper/2502.15777/) (2025)

## Abstract (arXiv)

AlphaZero has achieved impressive performance in deep reinforcement learning by utilizing an architecture that combines search and training of a neural network in self-play. Many researchers are looking for ways to reproduce and improve results for other games/tasks. However, the architecture is designed to learn from scratch, tabula rasa, accepting a cold-start problem in self-play. Recently, a warm-start enhancement method for Monte Carlo Tree Search was proposed to improve the self-play starting phase. It employs a fixed parameter I^prime to control the warm-start length. Improved performance was reported in small board games. In this paper we present results with an adaptive switch method. Experiments show that our approach works better than the fixed I^prime, especially for "deep," tactical, games (Othello and Connect Four). We conjecture that the adaptive value for I^prime is also influenced by the size of the game, and that on average I^prime will increase with game size. We conclude that AlphaZero-like deep reinforcement learning benefits from adaptive rollout based warm-start, as Rapid Action Value Estimate did for rollout-based reinforcement learning 15 years ago.