mlsanityData to 7 Oct 2026

Zero-Shot Offline Imitation Learning via Optimal Transport

arXiv 2410.08751 ↗First posted 11 Oct 2024cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Zero-shot imitation learning algorithms hold the promise of reproducing unseen behavior from as little as a single demonstration at test time. Existing practical approaches view the expert demonstration as a sequence of goals, enabling imitation with a high-level goal selector, and a low-level goal-conditioned policy. However, this framework can suffer from myopic behavior: the agent's immediate actions towards achieving individual goals may undermine long-term objectives. We introduce a novel method that mitigates this issue by directly optimizing the occupancy matching objective that is intrinsic to imitation learning. We propose to lift a goal-conditioned value function to a distance between occupancies, which are in turn approximated via a learned world model. The resulting method can learn from offline, suboptimal data, and is capable of non-myopic, zero-shot imitation, as we demonstrate in complex, continuous benchmarks. The code is available at https://github.com/martius-lab/zilot.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 55 of its 73 references that are in the map. First posted on arXiv 11 Oct 2024.

Builds on

55 of 73 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transport30 Jun
  2. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching4 Feb
  3. 2021Implicit Behavioral Cloning1 Sep
  4. 2021Adversarial Intrinsic Motivation for Reinforcement Learning27 May
  5. 2020Offline Learning from Demonstrations and Unlabeled Experience27 Nov
  6. 2020Primal Wasserstein Imitation Learning8 Jun
  7. 2018Behavioral Cloning from Observation4 May

Other references · 48

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2018World Modelscited by 1,444
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  11. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  12. 2017Hindsight Experience Replaycited by 1,096
36 more
  1. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  2. 2022A Generalist Agentcited by 802
  3. 2018Computational Optimal Transportcited by 765
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  5. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  7. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  8. 2019Self-Supervised Exploration via Disagreementcited by 263
  9. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  10. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  11. 2017One-Shot Imitation Learningcited by 232
  12. 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 138
  13. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  14. 2021Discovering and Achieving Goals via World Modelscited by 131
  15. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  16. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  17. 2018Zero-Shot Visual Imitationcited by 114
  18. 2021Learning One Representation to Optimize All Rewardscited by 110
  19. 2022Optimal Transport Tools (OTT): A JAX Toolbox for all things Wassersteincited by 96
  20. 2020Sample-efficient Cross-Entropy Method for Real-time Planningcited by 90
  21. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  22. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  23. 2019Sinkhorn Divergences for Unbalanced Optimal Transportcited by 52
  24. 2023Optimal Transport for Offline Imitation Learningcited by 42
  25. 2020Model-Based Visual Planning with Self-Supervised Functional Distancescited by 41
  26. 2021MBRL-Lib: A Modular Library for Model-based Reinforcement Learningcited by 41
  27. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  28. 2022Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrationscited by 30
  29. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 29
  30. 2022Meta Optimal Transportcited by 28
  31. 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21
  32. 2024Neural Optimal Transport with Lagrangian Costscited by 16
  33. 2024Global Reinforcement Learning: Beyond Linear and Convex Rewards via Submodular Semi-gradient Methodscited by 15
  34. 2019Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigationcited by 12
  35. 2023Action Inference by Maximising Evidence: Zero-Shot Imitation from Observation with World Modelscited by 5
  36. 2023Goal-conditioned Offline Planning from Curious Explorationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2026Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilitiescited by 2
  2. 2025HOTA: Hamiltonian framework for Optimal Transport Advectioncited by 1
  3. 2025On Discovering Algorithms for Adversarial Imitation Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Consistent Zero-Shot Imitation with Contrastive Goal Inference
  2. 2026Zero-shot Imitation Learning by Latent Topology Mapping
  3. 2024Align Your Intents: Offline Imitation Learning via Optimal Transport
  4. 2024Offline Imitation Learning by Controlling the Effective Planning Horizon
  5. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation
  6. 2025Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations
  7. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching
  8. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning