mlsanityData to 7 Oct 2026

Imitation Learning from Observation through Optimal Transport

arXiv 2310.01632 ↗First posted 2 Oct 2023cs.RO cs.AI cs.LGcited by 7

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning from Observation (ILfO) is a setting in which a learner tries to imitate the behavior of an expert, using only observational data and without the direct guidance of demonstrated actions. In this paper, we re-examine optimal transport for IL, in which a reward is generated based on the Wasserstein distance between the state trajectories of the learner and expert. We show that existing methods can be simplified to generate a reward function without requiring learned models or adversarial learning. Unlike many other state-of-the-art methods, our approach can be integrated with any RL algorithm and is amenable to ILfO. We demonstrate the effectiveness of this simple approach on a variety of continuous control tasks and find that it surpasses the state of the art in the IlfO setting, achieving expert-level performance across a range of evaluation domains even when observing only a single expert trajectory without actions.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 42 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

32 of 42 references.

Closest ancestors · 16

Same direction, newest first

  1. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transport30 Jun
  2. 2022IL-flOw: Imitation Learning from Observation using Normalizing Flows19 May
  3. 2021Imitation by Predicting Observations8 Jul
  4. 2021Adversarial Intrinsic Motivation for Reinforcement Learning27 May
  5. 2021Off-Policy Imitation Learning from Observations25 Feb
  6. 2021MobILE: Model-Based Imitation Learning From Observation Alone22 Feb
  7. 2020Imitation Learning with Sinkhorn Distances20 Aug
  8. 2020Primal Wasserstein Imitation Learning8 Jun
  9. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration5 Jun
  10. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  11. 2019Wasserstein Adversarial Imitation Learning19 Jun
  12. 2019Imitation Learning as f-Divergence Minimization30 May
4 more
  1. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  3. 2018Generative Adversarial Imitation from Observation17 Jul
  4. 2018Behavioral Cloning from Observation4 May

Other references · 16

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  6. 2016OpenAI Gymcited by 1,684
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  11. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  12. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
4 more
  1. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  2. 2020An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replaycited by 41
  3. 2021Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)cited by 33
  4. 2020An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 3; 2026: 2.

7 papers.

  1. 2023Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensorcited by 9
  2. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  3. 2023Generalizable Imitation Learning Through Pre-Trained Representationscited by 3
  4. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0
  5. 2025Noise-Guided Transport for Imitation Learningcited by 0
  6. 2026Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transportcited by 0
  7. 2026Differentiating Bisimulation Metrics: A Framework for Parametric Markov Chain Fitting via Bicausal Optimal Transportcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning from Observation: A Survey of Recent Advances
  2. 2024Align Your Intents: Offline Imitation Learning via Optimal Transport
  3. 2023Imitation Learning from Observation with Automatic Discount Scheduling
  4. 2022Imitation Learning from Observations under Transition Model Disparity
  5. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
  6. 2023On Combining Expert Demonstrations in Imitation Learning via Optimal Transport
  7. 2024Diffusion Imitation from Observation
  8. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement