# Structured Imitation Learning of Interactive Policies through Inverse Games

arXiv: https://arxiv.org/abs/2511.12848 · HTML: https://mlsanity.com/paper/2511.12848/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 13 of 13)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matching](https://mlsanity.com/paper/2409.07343/) (2024)
- [Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings](https://mlsanity.com/paper/1806.02813/) (2018)
- [Efficient Iterative Linear-Quadratic Approximations for Nonlinear Multi-Player General-Sum Differential Games](https://mlsanity.com/paper/1909.04694/) (2019)
- [Learning Human-to-Robot Handovers from Point Clouds](https://mlsanity.com/paper/2303.17592/) (2023)
- [Aligning Robot and Human Representations](https://mlsanity.com/paper/2302.01928/) (2023)
- [Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations](https://mlsanity.com/paper/2206.04632/) (2022)
- [BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay](https://mlsanity.com/paper/2402.14194/) (2024)
- [Mixed Strategy Nash Equilibrium for Crowd Navigation](https://mlsanity.com/paper/2403.01537/) (2024)
- [Inverse Mixed Strategy Games with Generative Trajectory Models](https://mlsanity.com/paper/2502.03356/) (2025)
- [Autonomous Human-Robot Interaction via Operator Imitation](https://mlsanity.com/paper/2504.02724/) (2025)

## Abstract (arXiv)

Generative model-based imitation learning methods have recently achieved strong results in learning high-complexity motor skills from human demonstrations. However, imitation learning of interactive policies that coordinate with humans in shared spaces without explicit communication remains challenging, due to the significantly higher behavioral complexity in multi-agent interactions compared to non-interactive tasks. In this work, we introduce a structured imitation learning framework for interactive policies by combining generative single-agent policy learning with a flexible yet expressive game-theoretic structure. Our method explicitly separates learning into two steps: first, we learn individual behavioral patterns from multi-agent demonstrations using standard imitation learning; then, we structurally learn inter-agent dependencies by solving an inverse game problem. Preliminary results in a synthetic 5-agent social navigation task show that our method significantly improves non-interactive policies and performs comparably to the ground truth interactive policy using only 50 demonstrations. These results highlight the potential of structured imitation learning in interactive settings.