# Interactive Inverse Reinforcement Learning for Cooperative Games

arXiv: https://arxiv.org/abs/2111.04698 · HTML: https://mlsanity.com/paper/2111.04698/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 7 of 7)

- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Reward-rational (implicit) choice: A unifying formalism for reward learning](https://mlsanity.com/paper/2002.04833/) (2020)
- [The MAGICAL Benchmark for Robust Imitation](https://mlsanity.com/paper/2011.00401/) (2020)
- [Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications](https://mlsanity.com/paper/1805.07687/) (2018)
- [Interactive Teaching Algorithms for Inverse Reinforcement Learning](https://mlsanity.com/paper/1905.11867/) (2019)
- [Learning to Collaborate in Markov Decision Processes](https://mlsanity.com/paper/1901.08029/) (2019)
- [Towards Deployment of Robust AI Agents for Human-Machine Partnerships](https://mlsanity.com/paper/1910.02330/) (2019)

## Built on by (most cited first, 7 of 7)

- [Environment Design for Inverse Reinforcement Learning](https://mlsanity.com/paper/2210.14972/) (2022)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioning](https://mlsanity.com/paper/2308.08427/) (2023)
- [PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learning](https://mlsanity.com/paper/2508.03693/) (2025)
- [Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning](https://mlsanity.com/paper/2607.08647/) (2026)
- [Interactive Inverse Reinforcement Learning of Interaction Scenarios via Bi-level Optimization](https://mlsanity.com/paper/2605.08131/) (2026)
- [A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learning](https://mlsanity.com/paper/2607.14373/) (2026)

## Abstract (arXiv)

We study the problem of designing autonomous agents that can learn to cooperate effectively with a potentially suboptimal partner while having no access to the joint reward function. This problem is modeled as a cooperative episodic two-agent Markov decision process. We assume control over only the first of the two agents in a Stackelberg formulation of the game, where the second agent is acting so as to maximise expected utility given the first agent's policy. How should the first agent act in order to learn the joint reward function as quickly as possible and so that the joint policy is as close to optimal as possible? We analyse how knowledge about the reward function can be gained in this interactive two-agent scenario. We show that when the learning agent's policies have a significant effect on the transition function, the reward function can be learned efficiently.