# Monte Carlo Q-learning for General Game Playing

arXiv: https://arxiv.org/abs/1802.05944 · HTML: https://mlsanity.com/paper/1802.05944/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2018.

## Built on by (most cited first, 2 of 2)

- [A Technique to Create Weaker Abstract Board Game Agents via Reinforcement Learning](https://mlsanity.com/paper/2209.00711/) (2022)
- [Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search](https://mlsanity.com/paper/2410.11642/) (2024)

## Abstract (arXiv)

After the recent groundbreaking results of AlphaGo, we have seen a strong interest in reinforcement learning in game playing. General Game Playing (GGP) provides a good testbed for reinforcement learning. In GGP, a specification of games rules is given. GGP problems can be solved by reinforcement learning. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee & Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex), to allow comparison to Banerjee et al. As expected, Q-learning converges, although much slower than MCTS. Borrowing an idea from MCTS, we enhance Q-learning with Monte Carlo Search, to give QM-learning. This enhancement improves the performance of pure Q-learning. We believe that QM-learning can also be used to improve performance of reinforcement learning further for larger games, something which we will test in future work.