mlsanityData to 7 Oct 2026

Watch and Match: Supercharging Imitation with Regularized Optimal Transport

arXiv 2206.15469 ↗First posted 30 Jun 2022cs.RO cs.AI cs.CVcited by 64

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning holds tremendous promise in learning policies efficiently for complex decision making problems. Current state-of-the-art algorithms often use inverse reinforcement learning (IRL), where given a set of expert demonstrations, an agent alternatively infers a reward function and the associated optimal policy. However, such IRL approaches often require substantial online interactions for complex control problems. In this work, we present Regularized Optimal Transport (ROT), a new imitation learning algorithm that builds on recent advances in optimal transport based trajectory-matching. Our key technical insight is that adaptively combining trajectory-matching rewards with behavior cloning can significantly accelerate imitation even with only a few demonstrations. Our experiments on 20 visual control tasks across the DeepMind Control Suite, the OpenAI Robotics Suite, and the Meta-World Benchmark demonstrate an average of 7.8X faster imitation to reach 90% of expert performance compared to prior state-of-the-art methods. On real-world robotic manipulation, with just one demonstration and an hour of online training, ROT achieves an average success rate of 90.1% across 14 tasks.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 43 of its 81 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

43 of 81 references.

Closest ancestors · 11

Same direction, newest first

  1. 2021Cross-Domain Imitation Learning via Optimal Transport7 Oct
  2. 2021Visual Adversarial Imitation Learning using Variational Models16 Jul
  3. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraints8 Mar
  4. 2020The MAGICAL Benchmark for Robust Imitation1 Nov
  5. 2020Imitation Learning with Sinkhorn Distances20 Aug
  6. 2020Primal Wasserstein Imitation Learning8 Jun
  7. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  8. 2019Wasserstein Adversarial Imitation Learning19 Jun
  9. 2019Recent Advances in Imitation Learning from Observation30 May
  10. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  11. 2018Generative Adversarial Imitation from Observation17 Jul

Other references · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2016OpenAI Gymcited by 1,684
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2016End to End Learning for Self-Driving Carscited by 1,000
  11. 2018DeepMind Control Suitecited by 935
  12. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
20 more
  1. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  2. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  3. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  4. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  5. 2018Learning Dexterous In-Hand Manipulationcited by 643
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  10. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 265
  11. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  12. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  13. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  14. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
  15. 2020Visual Imitation Made Easycited by 111
  16. 2022Jump-Start Reinforcement Learningcited by 102
  17. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  18. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  19. 2020CO-Optimal Transportcited by 42
  20. 2020Aligning Time Series on Incomparable Spacescited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 22; 2024: 20; 2025: 11; 2026: 8.

64 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  2. 2023On Bringing Robots Homecited by 101
  3. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  4. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  5. 2023QUAR-VLA: Vision-Language-Action Model for Quadruped Robotscited by 55
  6. 2023Imitation Bootstrapped Reinforcement Learningcited by 54
  7. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
  8. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 51
  9. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  10. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  11. 2023See to Touch: Learning Tactile Dexterity through Visual Incentivescited by 33
  12. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 30
38 more
  1. 2023Scalable Optimal Transport Methods in Machine Learning: A Contemporary Surveycited by 27
  2. 2024Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewardscited by 25
  3. 2025Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Correctionscited by 25
  4. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  5. 2022Planning for Sample Efficient Imitation Learningcited by 20
  6. 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 18
  7. 2024So You Think You Can Scale Up Autonomous Robot Data Collection?cited by 17
  8. 2023Visual Imitation Learning with Patch Rewardscited by 16
  9. 2023Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Owncited by 16
  10. 2024P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policiescited by 16
  11. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Searchcited by 16
  12. 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representationscited by 15
  13. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  14. 2023What Matters to You? Towards Visual Representation Alignment for Robot Learningcited by 14
  15. 2025Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directionscited by 12
  16. 2025Reinforcement Learning via Implicit Imitation Guidancecited by 12
  17. 2024PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinementcited by 11
  18. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  19. 2023Imitation Learning from Observation with Automatic Discount Schedulingcited by 10
  20. 2023PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulationcited by 9
  21. 2023Robust Visual Imitation Learning with Inverse Dynamics Representationscited by 8
  22. 2024Rethinking Optimal Transport in Offline Reinforcement Learningcited by 8
  23. 2024Robot Policy Learning with Temporal Optimal Transport Rewardcited by 8
  24. 2026From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuningcited by 8
  25. 2023Imitation Learning from Observation through Optimal Transportcited by 7
  26. 2024Align Your Intents: Offline Imitation Learning via Optimal Transportcited by 7
  27. 2024Constrained Ensemble Exploration for Unsupervised Skill Discoverycited by 7
  28. 2025Imitation Learning from a Single Temporally Misaligned Videocited by 7
  29. 2024ENOT: Expectile Regularization for Fast and Accurate Training of Neural Optimal Transportcited by 6
  30. 2024KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidancecited by 6
  31. 2025MORSE-500: A Programmatically Controllable Video Benchmark to Stress-Test Multimodal Reasoningcited by 6
  32. 2023Mimicking Better by Matching the Approximate Action Distributioncited by 5
  33. 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 5
  34. 2022Hierarchical Policy Blending As Optimal Transportcited by 4
  35. 2023Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environmentscited by 3
  36. 2024Zero-Shot Offline Imitation Learning via Optimal Transportcited by 3
  37. 2025The Temporal Trap: Entanglement in Pre-Trained Visual Representations for Visuomotor Policy Learningcited by 3
  38. 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learningcited by 2

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
  2. 2022Imitation Learning by Estimating Expertise of Demonstrators
  3. 2023On Combining Expert Demonstrations in Imitation Learning via Optimal Transport
  4. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  5. 2018An Algorithmic Perspective on Imitation Learning
  6. 2026Difference-Aware Retrieval Policies for Imitation Learning
  7. 2024Offline Imitation Learning Through Graph Search and Retrieval
  8. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards