On Combining Expert Demonstrations in Imitation Learning via Optimal Transport
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
Imitation learning (IL) seeks to teach agents specific tasks through expert demonstrations. One of the key approaches to IL is to define a distance between agent and expert and to find an agent policy that minimizes that distance. Optimal transport methods have been widely used in imitation learning as they provide ways to measure meaningful distances between agent and expert trajectories. However, the problem of how to optimally combine multiple expert demonstrations has not been widely studied. The standard method is to simply concatenate state (-action) trajectories, which is problematic when trajectories are multi-modal. We propose an alternative method that uses a multi-marginal optimal transport distance and enables the combination of multiple and diverse state-trajectories in the OT sense, providing a more sensible geometric average of the demonstrations. Our approach enables an agent to learn from several experts, and its efficiency is analyzed on OpenAI Gym control environments and demonstrates that the standard method is not always optimal.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 24 references that are in the map. First posted on arXiv 20 Jul 2023.
Builds on
16 of 24 references.
Closest ancestors · 5
Same direction, newest first
- 2021Implicit Behavioral Cloning1 Sep
- 2020Compressed imitation learning18 Sep
- 2020Imitation Learning with Sinkhorn Distances20 Aug
- 2020Primal Wasserstein Imitation Learning8 Jun
- 2019Wasserstein Adversarial Imitation Learning19 Jun
Other references · 11
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2021Transformer-based deep imitation learning for dual-arm robot manipulationcited by 32
- 2021VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigationcited by 31
- 2020TextGAIL: Generative Adversarial Imitation Learning for Text Generationcited by 28
- 2019Simultaneous Translation with Flexible Policy via Restricted Imitation Learningcited by 22
- 2020Aligning Time Series on Incomparable Spacescited by 16
- 2021Offline Inverse Reinforcement Learningcited by 14
- 2021Deep Adaptive Multi-Intention Inverse Reinforcement Learningcited by 6
- 2021Sliced Multi-Marginal Optimal Transportcited by 5
- 2019Imitation Learning of Robot Policies by Combining Language, Vision and Demonstrationcited by 3
- 2021A Generalised Inverse Reinforcement Learning Frameworkcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Optimal Transport Meets Reinforcement Learning: A Surveycited by 0
- 2026Optimal Transport Meets Reinforcement Learning: A Survey1 Oct 2026
Similar, not cited
Close in wording, no citation either way
- 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transport
- 2023Imitation Learning from Observation through Optimal Transport
- 2024Align Your Intents: Offline Imitation Learning via Optimal Transport
- 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Data
- 2022Imitation Learning by Estimating Expertise of Demonstrators
- 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamics
- 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning
- 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations