# Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations

arXiv: https://arxiv.org/abs/2205.02959 · HTML: https://mlsanity.com/paper/2205.02959/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 5 May 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations](https://mlsanity.com/paper/1904.06387/) (2019)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [Learning from Suboptimal Demonstration via Self-Supervised Reward Regression](https://mlsanity.com/paper/2010.11723/) (2020)
- [Coordinated Multi-Agent Imitation Learning](https://mlsanity.com/paper/1703.03121/) (2017)
- [Multi-Agent Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1907.13220/) (2019)
- [Multi-Agent Imitation Learning for Driving Simulation](https://mlsanity.com/paper/1803.01044/) (2018)
- [Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality](https://mlsanity.com/paper/2110.14754/) (2021)
- [TRAIL: Near-Optimal Imitation Learning with Suboptimal Data](https://mlsanity.com/paper/2110.14770/) (2021)
- [Generative Modeling of Multimodal Multi-Human Behavior](https://mlsanity.com/paper/1803.02015/) (2018)
- [Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning](https://mlsanity.com/paper/1903.05766/) (2019)
- [Learning Belief Representations for Imitation Learning in POMDPs](https://mlsanity.com/paper/1906.09510/) (2019)
- [Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Games](https://mlsanity.com/paper/1403.6508/) (2014)
- [Adversarial Imitation Learning from Incomplete Demonstrations](https://mlsanity.com/paper/1905.12310/) (2019)
- [Multi-Agent Interactions Modeling with Correlated Policies](https://mlsanity.com/paper/2001.03415/) (2020)
- [Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations](https://mlsanity.com/paper/1801.02124/) (2018)
- [Towards an AI Coach to Infer Team Mental Model Alignment in Healthcare](https://mlsanity.com/paper/2102.08507/) (2021)

## Built on by (most cited first, 2 of 2)

- [Automated Task-Time Interventions to Improve Teamwork using Imitation Learning](https://mlsanity.com/paper/2303.00413/) (2023)
- [Open Human-Robot Collaboration using Decentralized Inverse Reinforcement Learning](https://mlsanity.com/paper/2410.01790/) (2024)

## Abstract (arXiv)

We present Bayesian Team Imitation Learner (BTIL), an imitation learning algorithm to model the behavior of teams performing sequential tasks in Markovian domains. In contrast to existing multi-agent imitation learning techniques, BTIL explicitly models and infers the time-varying mental states of team members, thereby enabling learning of decentralized team policies from demonstrations of suboptimal teamwork. Further, to allow for sample- and label-efficient policy learning from small datasets, BTIL employs a Bayesian perspective and is capable of learning from semi-supervised demonstrations. We demonstrate and benchmark the performance of BTIL on synthetic multi-agent tasks as well as a novel dataset of human-agent teamwork. Our experiments show that BTIL can successfully learn team policies from demonstrations despite the influence of team members' (time-varying and potentially misaligned) mental states on their behavior.