# An Evolutionary Framework for Connect-4 as Test-Bed for Comparison of Advanced Minimax, Q-Learning and MCTS

arXiv: https://arxiv.org/abs/2405.16595 · HTML: https://mlsanity.com/paper/2405.16595/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 20 references that are in the map.
- First posted on arXiv 26 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 2 of 2)

- [Assessing the Potential of Classical Q-learning in General Game Playing](https://mlsanity.com/paper/1810.06078/) (2018)
- [AlphaZero-Inspired Game Learning: Faster Training by Using MCTS Only at Test Time](https://mlsanity.com/paper/2204.13307/) (2022)

## Built on by (most cited first, 2 of 2)

- [AlphaViT: A flexible game-playing AI for multiple games and variable board sizes](https://mlsanity.com/paper/2408.13871/) (2024)
- [Animation, Verification and Visualisation of Prolog Transition Systems with ProB](https://mlsanity.com/paper/2607.21192/) (2026)

## Abstract (arXiv)

A major challenge in decision making domains with large state spaces is to effectively select actions which maximize utility. In recent years, approaches such as reinforcement learning (RL) and search algorithms have been successful to tackle this issue, despite their differences. RL defines a learning framework that an agent explores and interacts with. Search algorithms provide a formalism to search for a solution. However, it is often difficult to evaluate the performances of such approaches in a practical way. Motivated by this problem, we focus on one game domain, i.e., Connect-4, and develop a novel evolutionary framework to evaluate three classes of algorithms: RL, Minimax and Monte Carlo tree search (MCTS). The contribution of this paper is threefold: i) we implement advanced versions of these algorithms and provide a systematic comparison with their standard counterpart, ii) we develop a novel evaluation framework, which we call the Evolutionary Tournament, and iii) we conduct an extensive evaluation of the relative performance of each algorithm to compare our findings. We evaluate different metrics and show that MCTS achieves the best results in terms of win percentage, whereas Minimax and Q-Learning are ranked in second and third place, respectively, although the latter is shown to be the fastest to make a decision.