mlsanityData to 14 Oct 2026

Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation

arXiv 1710.04615 ↗First posted 12 Oct 2017cs.LG cs.ROcited by 289

Not on the map. Why?

Cited by 158 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Imitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe how consumer-grade Virtual Reality headsets and hand tracking hardware can be used to naturally teleoperate robots to perform complex tasks. We also describe how imitation learning can learn deep neural network policies (mapping from pixels to actions) that can acquire the demonstrated skills. Our experiments showcase the effectiveness of our approach for learning visuomotor skills.

Key facts. Cited by 289 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 3; 2019: 9; 2020: 13; 2021: 40; 2022: 47; 2023: 52; 2024: 63; 2025: 41; 2026: 21.

289 papers.

Sort:

By area:

  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  2. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  3. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  4. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  6. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  7. 2021Implicit Behavioral Cloningcited by 452
  8. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  9. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  10. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
  11. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  12. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
38 more
  1. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  2. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  3. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  4. 2022Interactive Language: Talking to Robots in Real Timecited by 218
  5. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  6. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  7. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  8. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 169
  9. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  10. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  11. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 154
  12. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  13. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
  14. 2023Getting aligned on representational alignmentcited by 143
  15. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  16. 2023Large Language Models for Robotics: A Surveycited by 133
  17. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 133
  18. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
  19. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 116
  20. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 112
  21. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 108
  22. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 107
  23. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  24. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 93
  25. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 90
  26. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 89
  27. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  28. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 87
  29. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 84
  30. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 83
  31. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 82
  32. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 78
  33. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  34. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 77
  35. 2023XSkill: Cross Embodiment Skill Discoverycited by 75
  36. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  37. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 69
  38. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 69

50 most cited of 289 All 289 (JSON)