# Assessing the Potential of Classical Q-learning in General Game Playing

arXiv: https://arxiv.org/abs/1810.06078 · HTML: https://mlsanity.com/paper/1810.06078/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2018.

## Built on by (most cited first, 5 of 5)

- [Adaptive Warm-Start MCTS in AlphaZero-like Deep Reinforcement Learning](https://mlsanity.com/paper/2105.06136/) (2021)
- [An Evolutionary Framework for Connect-4 as Test-Bed for Comparison of Advanced Minimax, Q-Learning and MCTS](https://mlsanity.com/paper/2405.16595/) (2024)
- [Analysis of Bluffing by DQN and CFR in Leduc Hold'em Poker](https://mlsanity.com/paper/2509.04125/) (2025)
- [Towards Tackling MaxSAT by Combining Nested Monte Carlo with Local Search](https://mlsanity.com/paper/2302.13225/) (2023)
- [Learning to Play: Reinforcement Learning and Games](https://mlsanity.com/paper/2610.17391/) (2026)

## Abstract (arXiv)

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee & Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)footnotesource code: https://github.com/wh1992v/ggp-rl, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the ε-greedy strategy, we propose a first enhancement, the dynamic ε algorithm. In addition, inspired by (Gelly & Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.