# MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games

arXiv: https://arxiv.org/abs/2310.11305 · HTML: https://mlsanity.com/paper/2310.11305/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents](https://mlsanity.com/paper/1709.06009/) (2017)
- [Mastering Atari Games with Limited Data](https://mlsanity.com/paper/2111.00210/) (2021)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Learning and Planning in Complex Action Spaces](https://mlsanity.com/paper/2104.06303/) (2021)
- [Accelerating Self-Play Learning in Go](https://mlsanity.com/paper/1902.10565/) (2019)
- [Muesli: Combining Improvements in Policy Optimization](https://mlsanity.com/paper/2104.06159/) (2021)
- [ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero](https://mlsanity.com/paper/1902.04522/) (2019)
- [LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios](https://mlsanity.com/paper/2310.08348/) (2023)
- [Accelerating and Improving AlphaZero Using Population Based Training](https://mlsanity.com/paper/2003.06212/) (2020)
- [Polygames: Improved Zero Learning](https://mlsanity.com/paper/2001.09832/) (2020)

## Built on by (most cited first, 14 of 14)

- [Game Solving with Online Fine-Tuning](https://mlsanity.com/paper/2311.07178/) (2023)
- [Reinforcement Learning in Strategy-Based and Atari Games: A Review of Google DeepMinds Innovations](https://mlsanity.com/paper/2502.10303/) (2025)
- [OptionZero: Planning with Learned Options](https://mlsanity.com/paper/2502.16634/) (2025)
- [Demystifying MuZero Planning: Interpreting the Learned Model](https://mlsanity.com/paper/2411.04580/) (2024)
- [Strength Estimation and Human-Like Strength Adjustment in Games](https://mlsanity.com/paper/2502.17109/) (2025)
- [Evaluating Game Difficulty in Tetris Block Puzzle](https://mlsanity.com/paper/2603.18994/) (2026)
- [Bridging Local and Global Knowledge via Transformer in Board Games](https://mlsanity.com/paper/2410.05347/) (2024)
- [The Othello AI Arena: Evaluating Intelligent Systems Through Limited-Time Adaptation to Unseen Boards](https://mlsanity.com/paper/2508.09292/) (2025)
- [Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Search](https://mlsanity.com/paper/2512.21648/) (2025)
- [Regret-Guided Search Control for Efficient Learning in AlphaZero](https://mlsanity.com/paper/2602.20809/) (2026)
- [MAPLE: Multi-State Aggregated Policy Evaluation for AlphaZero in Imperfect-Information Games](https://mlsanity.com/paper/2605.24139/) (2026)
- [WallZero: Mastering the Game of WallGo with Strategic Analysis](https://mlsanity.com/paper/2606.17847/) (2026)
- [Belief-Guided Decision Making with Uncertainty Gating in the Game of Go](https://mlsanity.com/paper/2607.26946/) (2026)
- [The Surprising Effectiveness of Approximate Value Iteration in Self-Play](https://mlsanity.com/paper/2609.09094/) (2026)

## Abstract (arXiv)

This paper presents MiniZero, a zero-knowledge learning framework that supports four state-of-the-art algorithms, including AlphaZero, MuZero, Gumbel AlphaZero, and Gumbel MuZero. While these algorithms have demonstrated super-human performance in many games, it remains unclear which among them is most suitable or efficient for specific tasks. Through MiniZero, we systematically evaluate the performance of each algorithm in two board games, 9x9 Go and 8x8 Othello, as well as 57 Atari games. For two board games, using more simulations generally results in higher performance. However, the choice of AlphaZero and MuZero may differ based on game properties. For Atari games, both MuZero and Gumbel MuZero are worth considering. Since each game has unique characteristics, different algorithms and simulations yield varying results. In addition, we introduce an approach, called progressive simulation, which progressively increases the simulation budget during training to allocate computation more efficiently. Our empirical results demonstrate that progressive simulation achieves significantly superior performance in two board games. By making our framework and trained models publicly available, this paper contributes a benchmark for future research on zero-knowledge learning algorithms, assisting researchers in algorithm selection and comparison against these zero-knowledge learning baselines. Our code and data are available at https://rlg.iis.sinica.edu.tw/papers/minizero.