# Imitation Learning with Sinkhorn Distances

arXiv: https://arxiv.org/abs/2008.09167 · HTML: https://mlsanity.com/paper/2008.09167/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 20 Aug 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 20 of 20)

- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [Imitation Learning via Off-Policy Distribution Matching](https://mlsanity.com/paper/1912.05032/) (2019)
- [Reinforcement and Imitation Learning for Diverse Visuomotor Skills](https://mlsanity.com/paper/1802.09564/) (2018)
- [Improving GANs Using Optimal Transport](https://mlsanity.com/paper/1803.05573/) (2018)
- [Imitation Learning as f-Divergence Minimization](https://mlsanity.com/paper/1905.12888/) (2019)
- [Primal Wasserstein Imitation Learning](https://mlsanity.com/paper/2006.04678/) (2020)
- [Recent Advances in Imitation Learning from Observation](https://mlsanity.com/paper/1905.13566/) (2019)
- [Wasserstein Adversarial Imitation Learning](https://mlsanity.com/paper/1906.08113/) (2019)
- [Sample-Efficient Imitation Learning via Generative Adversarial Nets](https://mlsanity.com/paper/1809.02064/) (2018)
- [Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration](https://mlsanity.com/paper/2006.03503/) (2020)
- [Sample-efficient Adversarial Imitation Learning from Observation](https://mlsanity.com/paper/1906.07374/) (2019)

## Built on by (most cited first, 30 of 30)

- [Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations](https://mlsanity.com/paper/2303.01497/) (2023)
- [Watch and Match: Supercharging Imitation with Regularized Optimal Transport](https://mlsanity.com/paper/2206.15469/) (2022)
- [Cross-Domain Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2110.03684/) (2021)
- [For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal](https://mlsanity.com/paper/2304.04591/) (2023)
- [What Matters to You? Towards Visual Representation Alignment for Robot Learning](https://mlsanity.com/paper/2310.07932/) (2023)
- [Diffusion Imitation from Observation](https://mlsanity.com/paper/2410.05429/) (2024)
- [Imitation Learning from Observation with Automatic Discount Scheduling](https://mlsanity.com/paper/2310.07433/) (2023)
- [MILES: Making Imitation Learning Easy with Self-Supervision](https://mlsanity.com/paper/2410.19693/) (2024)
- [Rethinking Optimal Transport in Offline Reinforcement Learning](https://mlsanity.com/paper/2410.14069/) (2024)
- [Imitation Learning from Observation through Optimal Transport](https://mlsanity.com/paper/2310.01632/) (2023)
- [Imitation Learning from a Single Temporally Misaligned Video](https://mlsanity.com/paper/2502.05397/) (2025)
- [ENOT: Expectile Regularization for Fast and Accurate Training of Neural Optimal Transport](https://mlsanity.com/paper/2403.03777/) (2024)
- [KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidance](https://mlsanity.com/paper/2408.02912/) (2024)
- [TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distance](https://mlsanity.com/paper/2509.26627/) (2025)
- [Failure Identification in Imitation Learning Via Statistical and Semantic Filtering](https://mlsanity.com/paper/2604.13788/) (2026)
- [Mimicking Better by Matching the Approximate Action Distribution](https://mlsanity.com/paper/2306.09805/) (2023)
- [Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environments](https://mlsanity.com/paper/2310.08841/) (2023)
- [Policy Contrastive Imitation Learning](https://mlsanity.com/paper/2307.02829/) (2023)
- [MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization](https://mlsanity.com/paper/2608.25864/) (2026)
- [Overcoming Knowledge Barriers: Online Imitation Learning from Visual Observation with Pretrained World Models](https://mlsanity.com/paper/2404.18896/) (2024)
- [Self-Augmented Robot Trajectory: Efficient Imitation Learning via Safe Self-augmentation with Demonstrator-annotated Precision](https://mlsanity.com/paper/2509.09893/) (2025)
- [Distributionally-Constrained Policy Optimization via Unbalanced Optimal Transport](https://mlsanity.com/paper/2102.07889/) (2021)
- [On Combining Expert Demonstrations in Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2307.10810/) (2023)
- [Expert or not? assessing data quality in offline reinforcement learning](https://mlsanity.com/paper/2510.12638/) (2025)
- [ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstration](https://mlsanity.com/paper/2606.09381/) (2026)
- [Transfer RL via the Undo Maps Formalism](https://mlsanity.com/paper/2211.14469/) (2022)
- [Distill Knowledge in Multi-task Reinforcement Learning with Optimal-Transport Regularization](https://mlsanity.com/paper/2309.15603/) (2023)
- [Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator](https://mlsanity.com/paper/2401.16772/) (2024)
- [On Generalization and Distributional Update for Mimicking Observations with Adequate Exploration](https://mlsanity.com/paper/2501.12785/) (2025)
- [Data-driven simulator of multi-animal behavior with unknown dynamics via offline and online reinforcement learning](https://mlsanity.com/paper/2510.10451/) (2025)

## Abstract (arXiv)

Imitation learning algorithms have been interpreted as variants of divergence minimization problems. The ability to compare occupancy measures between experts and learners is crucial in their effectiveness in learning from demonstrations. In this paper, we present tractable solutions by formulating imitation learning as minimization of the Sinkhorn distance between occupancy measures. The formulation combines the valuable properties of optimal transport metrics in comparing non-overlapping distributions with a cosine distance cost defined in an adversarially learned feature space. This leads to a highly discriminative critic network and optimal transport plan that subsequently guide imitation learning. We evaluate the proposed approach using both the reward metric and the Sinkhorn distance metric on a number of MuJoCo experiments. For the implementation and reproducing results please refer to the following repository https://github.com/gpapagiannis/sinkhorn-imitation.