# Theory of Mind for Deep Reinforcement Learning in Hanabi

arXiv: https://arxiv.org/abs/2101.09328 · HTML: https://mlsanity.com/paper/2101.09328/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 5 of 5)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1811.01458/) (2018)
- [QMDP-Net: Deep Learning for Planning under Partial Observability](https://mlsanity.com/paper/1703.06692/) (2017)

## Built on by (most cited first, 11 of 11)

- [Mind the gap: Challenges of deep learning approaches to Theory of Mind](https://mlsanity.com/paper/2203.16540/) (2022)
- [Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2307.01158/) (2023)
- [BOSS: A Benchmark for Human Belief Prediction in Object-context Scenarios](https://mlsanity.com/paper/2206.10665/) (2022)
- [A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents](https://mlsanity.com/paper/2204.02889/) (2022)
- [Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning](https://mlsanity.com/paper/2507.03682/) (2025)
- [Cogment: Open Source Framework For Distributed Multi-actor Training, Deployment & Operations](https://mlsanity.com/paper/2106.11345/) (2021)
- [The Yokai Learning Environment: Tracking Beliefs Over Space and Time](https://mlsanity.com/paper/2508.12480/) (2025)
- [Adaptive Theory of Mind for LLM-based Multi-Agent Coordination](https://mlsanity.com/paper/2603.16264/) (2026)
- [Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads](https://mlsanity.com/paper/2301.02593/) (2023)
- [Autonomous Vehicle Decision-Making Framework for Considering Malicious Behavior at Unsignalized Intersections](https://mlsanity.com/paper/2409.17162/) (2024)
- [Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapings](https://mlsanity.com/paper/2604.25076/) (2026)

## Abstract (arXiv)

The partially observable card game Hanabi has recently been proposed as a new AI challenge problem due to its dependence on implicit communication conventions and apparent necessity of theory of mind reasoning for efficient play. In this work, we propose a mechanism for imbuing Reinforcement Learning agents with a theory of mind to discover efficient cooperative strategies in Hanabi. The primary contributions of this work are threefold: First, a formal definition of a computationally tractable mechanism for computing hand probabilities in Hanabi. Second, an extension to conventional Deep Reinforcement Learning that introduces reasoning over finitely nested theory of mind belief hierarchies. Finally, an intrinsic reward mechanism enabled by theory of mind that incentivizes agents to share strategically relevant private knowledge with their teammates. We demonstrate the utility of our algorithm against Rainbow, a state-of-the-art Reinforcement Learning agent.