# Game Solving with Online Fine-Tuning

arXiv: https://arxiv.org/abs/2311.07178 · HTML: https://mlsanity.com/paper/2311.07178/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 13 Nov 2023.

## Builds on (most cited first, 8 of 8)

- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [HyperTree Proof Search for Neural Theorem Proving](https://mlsanity.com/paper/2205.11491/) (2022)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Are AlphaZero-like Agents Robust to Adversarial Perturbations?](https://mlsanity.com/paper/2211.03769/) (2022)
- [MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games](https://mlsanity.com/paper/2310.11305/) (2023)
- [A Novel Approach to Solving Goal-Achieving Problems for Board Games](https://mlsanity.com/paper/2112.02563/) (2021)
- [A Local-Pattern Related Look-Up Table](https://mlsanity.com/paper/2212.13922/) (2022)

## Built on by (most cited first, 3 of 3)

- [Semi-Strongly solved: a New Definition Leading Computer to Perfect Gameplay](https://mlsanity.com/paper/2411.01029/) (2024)
- [Solving 7x7 Killall-Go with Seki Database](https://mlsanity.com/paper/2411.05565/) (2024)
- [Relevance-Zone Reduction in Game Solving](https://mlsanity.com/paper/2510.00689/) (2025)

## Abstract (arXiv)

Game solving is a similar, yet more difficult task than mastering a game. Solving a game typically means to find the game-theoretic value (outcome given optimal play), and optionally a full strategy to follow in order to achieve that outcome. The AlphaZero algorithm has demonstrated super-human level play, and its powerful policy and value predictions have also served as heuristics in game solving. However, to solve a game and obtain a full strategy, a winning response must be found for all possible moves by the losing player. This includes very poor lines of play from the losing side, for which the AlphaZero self-play process will not encounter. AlphaZero-based heuristics can be highly inaccurate when evaluating these out-of-distribution positions, which occur throughout the entire search. To address this issue, this paper investigates applying online fine-tuning while searching and proposes two methods to learn tailor-designed heuristics for game solving. Our experiments show that using online fine-tuning can solve a series of challenging 7x7 Killall-Go problems, using only 23.54% of computation time compared to the baseline without online fine-tuning. Results suggest that the savings scale with problem size. Our method can further be extended to any tree search algorithm for problem solving. Our code is available at https://rlg.iis.sinica.edu.tw/papers/neurips2023-online-fine-tuning-solver.