# Neurohex: A Deep Q-learning Hex Agent

arXiv: https://arxiv.org/abs/1604.07097 · HTML: https://mlsanity.com/paper/1604.07097/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2016.

## Built on by (most cited first, 5 of 5)

- [Scaling Scaling Laws with Board Games](https://mlsanity.com/paper/2104.03113/) (2021)
- [HexCNN: A Framework for Native Hexagonal Convolutional Neural Networks](https://mlsanity.com/paper/2101.10897/) (2021)
- [CubeTR: Learning to Solve The Rubiks Cube Using Transformers](https://mlsanity.com/paper/2111.06036/) (2021)
- [Logic-based AI for Interpretable Board Game Winner Prediction with Tsetlin Machine](https://mlsanity.com/paper/2203.04378/) (2022)
- [Biologically inspired deep residual networks for computer vision applications](https://mlsanity.com/paper/2205.02551/) (2022)

## Abstract (arXiv)

DeepMind's recent spectacular success in using deep convolutional neural nets and machine learning to build superhuman level agents --- e.g. for Atari games via deep Q-learning and for the game of Go via Reinforcement Learning --- raises many questions, including to what extent these methods will succeed in other domains. In this paper we consider DQL for the game of Hex: after supervised initialization, we use selfplay to train NeuroHex, an 11-layer CNN that plays Hex on the 13x13 board. Hex is the classic two-player alternate-turn stone placement game played on a rhombus of hexagonal cells in which the winner is whomever connects their two opposing sides. Despite the large action and state space, our system trains a Q-network capable of strong play with no search. After two weeks of Q-learning, NeuroHex achieves win-rates of 20.4% as first player and 2.1% as second player against a 1-second/move version of MoHex, the current ICGA Olympiad Hex champion. Our data suggests further improvement might be possible with more training time.