# Deep Adaptive Multi-Intention Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.06692 · HTML: https://mlsanity.com/paper/2107.06692/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets](https://mlsanity.com/paper/1705.10479/) (2017)
- [Stick-Breaking Variational Autoencoders](https://mlsanity.com/paper/1605.06197/) (2016)
- [Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors](https://mlsanity.com/paper/1903.10304/) (2019)
- [Learning to Simulate on Sparse Trajectory Data](https://mlsanity.com/paper/2103.11845/) (2021)
- [Inverse Reinforce Learning with Nonparametric Behavior Clustering](https://mlsanity.com/paper/1712.05514/) (2017)
- [Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach](https://mlsanity.com/paper/1810.00159/) (2018)
- [On the Variational Posterior of Dirichlet Process Deep Latent Gaussian Mixture Models](https://mlsanity.com/paper/2006.08993/) (2020)

## Built on by (most cited first, 6 of 6)

- [Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2209.11908/) (2022)
- [Multi-intention Inverse Q-learning for Interpretable Behavior Representation](https://mlsanity.com/paper/2311.13870/) (2023)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning](https://mlsanity.com/paper/2602.07496/) (2026)
- [Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration](https://mlsanity.com/paper/2202.07014/) (2022)
- [On Combining Expert Demonstrations in Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2307.10810/) (2023)

## Abstract (arXiv)

This paper presents a deep Inverse Reinforcement Learning (IRL) framework that can learn an a priori unknown number of nonlinear reward functions from unlabeled experts' demonstrations. For this purpose, we employ the tools from Dirichlet processes and propose an adaptive approach to simultaneously account for both complex and unknown number of reward functions. Using the conditional maximum entropy principle, we model the experts' multi-intention behaviors as a mixture of latent intention distributions and derive two algorithms to estimate the parameters of the deep reward network along with the number of experts' intentions from unlabeled demonstrations. The proposed algorithms are evaluated on three benchmarks, two of which have been specifically extended in this study for multi-intention IRL, and compared with well-known baselines. We demonstrate through several experiments the advantages of our algorithms over the existing approaches and the benefits of online inferring, rather than fixing beforehand, the number of expert's intentions.