mlsanityData to 7 Oct 2026

Primal Wasserstein Imitation Learning

arXiv 2006.04678 ↗First posted 8 Jun 2020cs.LG stat.MLcited by 120

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning (IL) methods seek to match the behavior of an agent with that of an expert. In the present work, we propose a new IL method based on a conceptually simple algorithm: Primal Wasserstein Imitation Learning (PWIL), which ties to the primal form of the Wasserstein distance between the expert and the agent state-action distributions. We present a reward function which is derived offline, as opposed to recent adversarial IL algorithms that learn a reward function through interactions with the environment, and which requires little fine-tuning. We show that we can recover expert behavior on a variety of continuous control tasks of the MuJoCo domain in a sample efficient manner in terms of agent interactions and of expert interactions with the environment. Finally, we show that the behavior of the agent we train matches the behavior of the expert with the Wasserstein distance, rather than the commonly used proxy of performance.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 74 references that are in the map. First posted on arXiv 8 Jun 2020.

Builds on

36 of 74 references.

Closest ancestors · 10

Same direction, newest first

  1. 2019State Alignment-based Imitation Learning21 Nov
  2. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  3. 2019Wasserstein Adversarial Imitation Learning19 Jun
  4. 2019Imitation Learning as f-Divergence Minimization30 May
  5. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards27 May
  6. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May
  7. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  8. 2018Risk-Sensitive Generative Adversarial Imitation Learning13 Aug
  9. 2018Imitating Latent Policies from Observation21 May
  10. 2018Behavioral Cloning from Observation4 May

Other references · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Layer Normalizationcited by 5,579
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2016OpenAI Gymcited by 1,684
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  10. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  11. 2018Exploration by Random Network Distillationcited by 966
  12. 2018Computational Optimal Transportcited by 765
14 more
  1. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  2. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  3. 2018Learning Dexterous In-Hand Manipulationcited by 643
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  5. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  6. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  7. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  8. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  9. 2019Temporal Cycle-Consistency Learningcited by 151
  10. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  11. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  12. 2018Playing hard exploration games by watching YouTubecited by 98
  13. 2019Learning to Score Behaviors for Guided Policy Optimizationcited by 34
  14. 2018Reinforcement Learning with Wasserstein Distance Regularisation, with Applications to Multipolicy Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 3; 2021: 20; 2022: 19; 2023: 36; 2024: 19; 2025: 15; 2026: 8.

120 papers.

Sort:

By area:

  1. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  2. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  3. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  4. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 80
  5. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  6. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
  7. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  8. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  9. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  10. 2023Optimal Transport for Offline Imitation Learningcited by 42
  11. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  12. 2021RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learningcited by 31
38 more
  1. 2020Imitation Learning with Sinkhorn Distancescited by 30
  2. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
  3. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
  4. 2022Meta Optimal Transportcited by 28
  5. 2022Proximal Point Imitation Learningcited by 25
  6. 2021Wasserstein Unsupervised Reinforcement Learningcited by 21
  7. 2021Continuous Control with Action Quantization from Demonstrationscited by 21
  8. 2023Coherent Soft Imitation Learningcited by 21
  9. 2022Planning for Sample Efficient Imitation Learningcited by 20
  10. 2024Diffusion-Reward Adversarial Imitation Learningcited by 18
  11. 2021Hyperparameter Selection for Imitation Learningcited by 17
  12. 2021There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learningcited by 17
  13. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitionscited by 16
  14. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  15. 2024Imitation Learning in Discounted Linear MDPs without exploration assumptionscited by 15
  16. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  17. 2023What Matters to You? Towards Visual Representation Alignment for Robot Learningcited by 14
  18. 2024SEABO: A Simple Search-Based Method for Offline Imitation Learningcited by 14
  19. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  20. 2023Unsupervised Behavior Extraction via Random Intent Priorscited by 13
  21. 2022IL-flOw: Imitation Learning from Observation using Normalizing Flowscited by 12
  22. 2023A Coupled Flow Approach to Imitation Learningcited by 12
  23. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  24. 2022Adversarial Imitation Learning from Video using a State Observercited by 10
  25. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 10
  26. 2023Imitation Learning from Observation with Automatic Discount Schedulingcited by 10
  27. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
  28. 2020Regularized Inverse Reinforcement Learningcited by 9
  29. 2021A Pragmatic Look at Deep Imitation Learningcited by 9
  30. 2023PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulationcited by 9
  31. 2022Learning Energy Networks with Generalized Fenchel-Young Lossescited by 8
  32. 2023Robust Visual Imitation Learning with Inverse Dynamics Representationscited by 8
  33. 2024Rethinking Optimal Transport in Offline Reinforcement Learningcited by 8
  34. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  35. 2023Imitation Learning from Observation through Optimal Transportcited by 7
  36. 2023Imitate the Good and Avoid the Bad: An Incremental Approach to Safe Reinforcement Learningcited by 7
  37. 2024SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learningcited by 7
  38. 2024LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulationcited by 7

50 most cited of 120 All 120 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Imitation Learning by Reinforcement Learning
  2. 2019Reinforced Imitation in Heterogeneous Action Space
  3. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
  4. 2019A New Framework for Query Efficient Active Imitation Learning
  5. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
  6. 2024Reward-free World Models for Online Imitation Learning
  7. 2020State-only Imitation with Transition Dynamics Mismatch
  8. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances