# Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates

arXiv: https://arxiv.org/abs/2111.09800 · HTML: https://mlsanity.com/paper/2111.09800/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 18 Nov 2021.

## Builds on (most cited first, 9 of 9)

- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- ["Other-Play" for Zero-Shot Coordination](https://mlsanity.com/paper/2003.02979/) (2020)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Off-Belief Learning](https://mlsanity.com/paper/2103.04000/) (2021)
- [Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1912.02288/) (2019)
- [Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi](https://mlsanity.com/paper/2107.07630/) (2021)
- [Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings](https://mlsanity.com/paper/2106.09086/) (2021)
- [Instructive artificial intelligence (AI) for human training, assistance, and explainability](https://mlsanity.com/paper/2111.01726/) (2021)

## Abstract (arXiv)

In 2021 the Johns Hopkins University Applied Physics Laboratory held an internal challenge to develop artificially intelligent (AI) agents that could excel at the collaborative card game Hanabi. Agents were evaluated on their ability to play with human players whom the agents had never previously encountered. This study details the development of the agent that won the challenge by achieving a human-play average score of 16.5, outperforming the current state-of-the-art for human-bot Hanabi scores. The winning agent's development consisted of observing and accurately modeling the author's decision making in Hanabi, then training with a behavioral clone of the author. Notably, the agent discovered a human-complementary play style by first mimicking human decision making, then exploring variations to the human-like strategy that led to higher simulated human-bot scores. This work examines in detail the design and implementation of this human compatible Hanabi teammate, as well as the existence and implications of human-complementary strategies and how they may be explored for more successful applications of AI in human machine teams.