mlsanityData to 7 Oct 2026

Wasserstein Adversarial Imitation Learning

arXiv 1906.08113 ↗First posted 19 Jun 2019cs.LG stat.MLcited by 59

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularization. In this paper, we show a natural connection between inverse reinforcement learning approaches and Optimal Transport, that enables more general reward functions with desirable properties (e.g., smoothness). Based on our observation, we propose a novel approach called Wasserstein Adversarial Imitation Learning. Our approach considers the Kantorovich potentials as a reward function and further leverages regularized optimal transport to enable large-scale applications. In several robotic experiments, our approach outperforms the baselines in terms of average cumulative rewards and shows a significant improvement in sample-efficiency, by requiring just one expert demonstration.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 39 references that are in the map. First posted on arXiv 19 Jun 2019.

Builds on

16 of 39 references.

References · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  4. 2017Improved Training of Wasserstein GANscited by 2,223
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2016OpenAI Gymcited by 1,684
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2018Computational Optimal Transportcited by 765
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  11. 2016Stochastic Optimization for Large-scale Optimal Transportcited by 188
  12. 2017Large-Scale Optimal Transport and Mapping Estimationcited by 120
4 more
  1. 2017Smooth and Sparse Optimal Transportcited by 83
  2. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  3. 2017Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learningcited by 50
  4. 2018Learning to Optimize via Wasserstein Deep Inverse Optimal Controlcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 7; 2021: 11; 2022: 4; 2023: 16; 2024: 7; 2025: 10; 2026: 3.

59 papers.

Sort:

By area:

  1. 2020Primal Wasserstein Imitation Learningcited by 120
  2. 2021Reward is enough for convex MDPscited by 82
  3. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  4. 2019State Alignment-based Imitation Learningcited by 73
  5. 2021Off-Policy Imitation Learning from Observationscited by 70
  6. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
  7. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  8. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  9. 2023Behavior Proximal Policy Optimizationcited by 43
  10. 2023Optimal Transport for Offline Imitation Learningcited by 42
  11. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  12. 2023See to Touch: Learning Tactile Dexterity through Visual Incentivescited by 33
38 more
  1. 2020Imitation Learning with Sinkhorn Distancescited by 30
  2. 2021Online Apprenticeship Learningcited by 28
  3. 2020Robust Imitation Learning from Noisy Demonstrationscited by 26
  4. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamicscited by 26
  5. 2022Proximal Point Imitation Learningcited by 25
  6. 2023Energy-guided Entropic Neural Optimal Transportcited by 25
  7. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  8. 2020Non-Adversarial Imitation Learning and its Connections to Adversarial Methodscited by 16
  9. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learningcited by 16
  10. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  11. 2023What Matters to You? Towards Visual Representation Alignment for Robot Learningcited by 14
  12. 2025Generative Models in Decision Making: A Surveycited by 14
  13. 2023Unsupervised Behavior Extraction via Random Intent Priorscited by 13
  14. 2024Diffusion Imitation from Observationcited by 12
  15. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
  16. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulationcited by 8
  17. 2023Imitation Learning from Observation through Optimal Transportcited by 7
  18. 2025Imitation Learning from a Single Temporally Misaligned Videocited by 7
  19. 2023Provably Convergent Policy Optimization via Metric-aware Trust Region Methodscited by 6
  20. 2024KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidancecited by 6
  21. 2024Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignmentcited by 6
  22. 2025Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrationscited by 6
  23. 2021Wasserstein Distance Maximizing Intrinsic Controlcited by 5
  24. 2022Learning from Imperfect Demonstrations via Adversarial Confidence Transfercited by 5
  25. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  26. 2024Enhancing Reinforcement Learning Agents with Local Guidescited by 4
  27. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learningcited by 3
  28. 2023Policy Contrastive Imitation Learningcited by 3
  29. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theorycited by 3
  30. 2021On the Benefits of Inducing Local Lipschitzness for Robust Generative Adversarial Imitation Learningcited by 2
  31. 2022Fail-Safe Adversarial Generative Imitation Learningcited by 2
  32. 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learningcited by 2
  33. 2025Constrained Sliced Wasserstein Embeddingcited by 2
  34. 2025FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learningcited by 2
  35. 2020Inverse Reinforcement Learning via Matching of Optimality Profilescited by 1
  36. 2021Semi-Supervised Imitation Learning with Mixed Qualities of Demonstrations for Autonomous Drivingcited by 1
  37. 2025From Novelty to Imitation: Self-Distilled Rewards for Offline Reinforcement Learningcited by 1
  38. 2025On Discovering Algorithms for Adversarial Imitation Learningcited by 1

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
  2. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  3. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  4. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
  5. 2023Regularizing Adversarial Imitation Learning Using Causal Invariance
  6. 2022Robust Learning from Observation with Model Misspecification
  7. 2020ADAIL: Adaptive Adversarial Imitation Learning
  8. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms