mlsanityData to 7 Oct 2026

Align Your Intents: Offline Imitation Learning via Optimal Transport

arXiv 2402.13037 ↗First posted 20 Feb 2024cs.LG cs.AIcited by 7

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Offline Reinforcement Learning (RL) addresses the problem of sequential decision-making by learning optimal policy through pre-collected data, without interacting with the environment. As yet, it has remained somewhat impractical, because one rarely knows the reward explicitly and it is hard to distill it retrospectively. Here, we show that an imitating agent can still learn the desired behavior merely from observing the expert, despite the absence of explicit rewards or action labels. In our method, AILOT (Aligned Imitation Learning via Optimal Transport), we involve special representation of states in a form of intents that incorporate pairwise spatial distances within the data. Given such representations, we define intrinsic reward function via optimal transport distance between the expert's and the agent's trajectories. We report that AILOT outperforms state-of-the art offline imitation learning algorithms on D4RL benchmarks and improves the performance of other offline RL algorithms by dense reward relabelling in the sparse-reward tasks.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 56 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

47 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transport30 Jun
  2. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching4 Feb
  3. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  4. 2020Offline Learning from Demonstrations and Unlabeled Experience27 Nov
  5. 2020Intrinsic Reward Driven Imitation Learning via Generative Model26 Jun
  6. 2018Generative Adversarial Imitation from Observation17 Jul

Other references · 41

  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  8. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  9. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  10. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  11. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  12. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
29 more
  1. 2019Efficient Exploration via State Marginal Matchingcited by 186
  2. 2021Equinox: neural networks in JAX via callable PyTrees and filtered transformationscited by 143
  3. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  4. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  5. 2022Neural Optimal Transportcited by 123
  6. 2018Improving GANs Using Optimal Transportcited by 122
  7. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  8. 2021Learning One Representation to Optimize All Rewardscited by 110
  9. 2018Universal Successor Features Approximatorscited by 102
  10. 2022Optimal Transport Tools (OTT): A JAX Toolbox for all things Wassersteincited by 96
  11. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  12. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  13. 2021Generative Modeling with Optimal Transport Mapscited by 80
  14. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  15. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  16. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  17. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  18. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 61
  19. 2023Optimal Transport for Offline Imitation Learningcited by 42
  20. 2023Contrastive Difference Predictive Codingcited by 35
  21. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 31
  22. 2019Deep Bayesian Reward Learning from Preferencescited by 28
  23. 2023Accelerating Exploration with Unlabeled Prior Datacited by 21
  24. 2023GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Modelscited by 20
  25. 2023CLUE: Calibrated Latent Guidance for Offline Reinforcement Learningcited by 11
  26. 2021Reinforcement Learning based Disease Progression Model for Alzheimer's Diseasecited by 10
  27. 2020BRUL`E: Barycenter-Regularized Unsupervised Landmark Extractioncited by 5
  28. 2021Landmarks Augmentation with Manifold-Barycentric Oversamplingcited by 4
  29. 2019Reinforcement learning for suppression of collective activity in oscillatory ensemblescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 1.

7 papers.

  1. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  2. 2025Imitation Learning from a Single Temporally Misaligned Videocited by 7
  3. 2024NOLO: Navigate Only Look Oncecited by 3
  4. 2025From Novelty to Imitation: Self-Distilled Rewards for Offline Reinforcement Learningcited by 1
  5. 2025HOTA: Hamiltonian framework for Optimal Transport Advectioncited by 1
  6. 2025Minimal Ingredients for Reward Assignment from Expert Demonstrationscited by 0
  7. 2026Latent Wasserstein Adversarial Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SEABO: A Simple Search-Based Method for Offline Imitation Learning
  2. 2023Imitation Learning from Observation through Optimal Transport
  3. 2019Reinforced Imitation in Heterogeneous Action Space
  4. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
  5. 2024Zero-Shot Offline Imitation Learning via Optimal Transport
  6. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learning
  7. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environments
  8. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism