# LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2106.01777 · HTML: https://mlsanity.com/paper/2106.01777/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 12 of 12)

- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [Statistical guarantees for the EM algorithm: From population to sample-based analysis](https://mlsanity.com/paper/1408.2156/) (2014)
- [Imitation Learning as f-Divergence Minimization](https://mlsanity.com/paper/1905.12888/) (2019)
- [Quantifying Differences in Reward Functions](https://mlsanity.com/paper/2006.13900/) (2020)
- [Meta-Inverse Reinforcement Learning with Probabilistic Context Variables](https://mlsanity.com/paper/1909.09314/) (2019)
- [OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1709.06683/) (2017)
- [Adjusting for Chance Clustering Comparison Measures](https://mlsanity.com/paper/1512.01286/) (2015)
- [Multi-task Maximum Entropy Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.08882/) (2018)
- [Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation](https://mlsanity.com/paper/2001.00503/) (2020)
- [Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms](https://mlsanity.com/paper/2012.00889/) (2020)
- [Sinkhorn EM: An Expectation-Maximization algorithm based on entropic optimal transport](https://mlsanity.com/paper/2006.16548/) (2020)

## Built on by (most cited first, 2 of 2)

- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning](https://mlsanity.com/paper/2602.07496/) (2026)

## Abstract (arXiv)

Multiple-Intent Inverse Reinforcement Learning (MI-IRL) seeks to find a reward function ensemble to rationalize demonstrations of different but unlabelled intents. Within the popular expectation maximization (EM) framework for learning probabilistic MI-IRL models, we present a warm-start strategy based on up-front clustering of the demonstrations in feature space. Our theoretical analysis shows that this warm-start solution produces a near-optimal reward ensemble, provided the behavior modes satisfy mild separation conditions. We also propose a MI-IRL performance metric that generalizes the popular Expected Value Difference measure to directly assesses learned rewards against the ground-truth reward ensemble. Our metric elegantly addresses the difficulty of pairing up learned and ground truth rewards via a min-cost flow formulation, and is efficiently computable. We also develop a MI-IRL benchmark problem that allows for more comprehensive algorithmic evaluations. On this problem, we find our MI-IRL warm-start strategy helps avoid poor quality local minima reward ensembles, resulting in a significant improvement in behavior clustering. Our extensive sensitivity analysis demonstrates that the quality of the learned reward ensembles is improved under various settings, including cases where our theoretical assumptions do not necessarily hold. Finally, we demonstrate the effectiveness of our methods by discovering distinct driving styles in a large real-world dataset of driver GPS trajectories.