# Warm-Start AlphaZero Self-Play Search Enhancements

arXiv: https://arxiv.org/abs/2004.12357 · HTML: https://mlsanity.com/paper/2004.12357/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2020.

## Built on by (most cited first, 5 of 5)

- [Adaptive Warm-Start MCTS in AlphaZero-like Deep Reinforcement Learning](https://mlsanity.com/paper/2105.06136/) (2021)
- [AlphaDDA: Strategies for Adjusting the Playing Strength of a Fully Trained AlphaZero System to a Suitable Human Training Partner](https://mlsanity.com/paper/2111.06266/) (2021)
- [AlphaZero-Inspired Game Learning: Faster Training by Using MCTS Only at Test Time](https://mlsanity.com/paper/2204.13307/) (2022)
- [Towards Tackling MaxSAT by Combining Nested Monte Carlo with Local Search](https://mlsanity.com/paper/2302.13225/) (2023)
- [AlphaViT: A flexible game-playing AI for multiple games and variable board sizes](https://mlsanity.com/paper/2408.13871/) (2024)

## Abstract (arXiv)

Recently, AlphaZero has achieved landmark results in deep reinforcement learning, by providing a single self-play architecture that learned three different games at super human level. AlphaZero is a large and complicated system with many parameters, and success requires much compute power and fine-tuning. Reproducing results in other games is a challenge, and many researchers are looking for ways to improve results while reducing computational demands. AlphaZero's design is purely based on self-play and makes no use of labeled expert data ordomain specific enhancements; it is designed to learn from scratch. We propose a novel approach to deal with this cold-start problem by employing simple search enhancements at the beginning phase of self-play training, namely Rollout, Rapid Action Value Estimate (RAVE) and dynamically weighted combinations of these with the neural network, and Rolling Horizon Evolutionary Algorithms (RHEA). Our experiments indicate that most of these enhancements improve the performance of their baseline player in three different (small) board games, with especially RAVE based variants playing strongly.