mlsanityData to 14 Oct 2026

Zero-Shot Visual Imitation

arXiv 1804.08606 ↗First posted 23 Apr 2018cs.LG cs.AI cs.CVcited by 117

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

The current dominant paradigm for imitation learning relies on strong supervision of expert actions to learn both 'what' and 'how' to imitate. We pursue an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its experience into a goal-conditioned skill policy with a novel forward consistency loss. In our framework, the role of the expert is only to communicate the goals (i.e., what to imitate) during inference. The learned policy is then employed to mimic the expert (i.e., how to imitate) after seeing just a sequence of images demonstrating the desired task. Our method is 'zero-shot' in the sense that the agent never has access to expert actions during training or for the task demonstration at inference. We evaluate our zero-shot imitator in two real-world settings: complex rope manipulation with a Baxter robot and navigation in previously unseen office environments with a TurtleBot. Through further experiments in VizDoom simulation, we provide evidence that better mechanisms for exploration lead to learning a more capable policy which in turn improves end task performance. Videos, models, and more details are available at https://pathak22.github.io/zeroshot-imitation/

Key facts. Cited by 117 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 23 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 10; 2020: 4; 2021: 22; 2022: 19; 2023: 24; 2024: 13; 2025: 13; 2026: 11.

117 papers.

Sort:

By area:

  1. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  2. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  3. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  4. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  5. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  6. 2024Navigation World Modelscited by 272
  7. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  8. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  9. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  10. 2022Human-to-Robot Imitation in the Wildcited by 161
  11. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  12. 2021Discovering and Achieving Goals via World Modelscited by 131
38 more
  1. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 107
  2. 2023Compositional Foundation Models for Hierarchical Planningcited by 95
  3. 2019Recent Advances in Imitation Learning from Observationcited by 93
  4. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 91
  5. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 89
  6. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  7. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 82
  8. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  9. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 78
  10. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  11. 2019State Alignment-based Imitation Learningcited by 73
  12. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 58
  13. 2021FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policycited by 54
  14. 2023Zero-Shot Robot Manipulation from Passive Human Videoscited by 51
  15. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 46
  16. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 46
  17. 2025DynaGuide: Steering Diffusion Polices with Active Dynamic Guidancecited by 44
  18. 2023GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videoscited by 40
  19. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 38
  20. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Modelcited by 37
  21. 2022K-VIL: Keypoints-based Visual Imitation Learningcited by 27
  22. 2021Differentiable Spatial Planning using Transformerscited by 25
  23. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  24. 2022ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentationcited by 19
  25. 2023Lifelong Robot Learning with Human Assisted Language Plannerscited by 19
  26. 2021Shaping embodied agent behavior with activity-context priors from egocentric videocited by 17
  27. 2021Object-Aware Regularization for Addressing Causal Confusion in Imitation Learningcited by 17
  28. 2023Visual Imitation Learning with Patch Rewardscited by 16
  29. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  30. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  31. 2019Generative predecessor models for sample-efficient imitation learningcited by 14
  32. 2020Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learningcited by 14
  33. 2023SIRL: Similarity-based Implicit Representation Learningcited by 14
  34. 2019Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigationcited by 13
  35. 2021Imitation by Predicting Observationscited by 13
  36. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  37. 2023Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedbackcited by 13
  38. 2024Resisting Stochastic Risks in Diffusion Planners with the Trajectory Aggregation Treecited by 13

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Zero-shot Imitation Learning by Latent Topology Mapping
  2. 2023Zero-Shot Transfer in Imitation Learning
  3. 2021Towards More Generalizable One-shot Visual Imitation Learning
  4. 2025OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generation
  5. 2024One-shot Imitation in a Non-Stationary Environment via Multi-Modal Skill
  6. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks
  7. 2021Manipulator-Independent Representations for Visual Imitation
  8. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search