mlsanityData to 13 Oct 2026

Human-in-the-Loop Imitation Learning using Remote Teleoperation

arXiv 2012.06733 ↗First posted 12 Dec 2020cs.RO cs.AI cs.LGcited by 106

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation Learning is a promising paradigm for learning complex robot manipulation skills by reproducing behavior from human demonstrations. However, manipulation tasks often contain bottleneck regions that require a sequence of precise actions to make meaningful progress, such as a robot inserting a pod into a coffee machine to make coffee. Trained policies can fail in these regions because small deviations in actions can lead the policy into states not covered by the demonstrations. Intervention-based policy learning is an alternative that can address this issue -- it allows human operators to monitor trained policies and take over control when they encounter failures. In this paper, we build a data collection system tailored to 6-DoF manipulation settings, that enables remote human operators to monitor and intervene on trained policies. We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators, and further show that our method outperforms multiple state-of-the-art baselines for learning from the human interventions on a challenging robot threading task and a coffee making task. Additional results and videos at https://sites.google.com/stanford.edu/iwr .

Key facts. Cited by 106 papers in the map (data through 13 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 43 references that are in the map. First posted on arXiv 12 Dec 2020.

Builds on

23 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  2. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov
  3. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills26 Feb

Other references · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2017Deep reinforcement learning from human preferencescited by 3,871
  3. 2016Generative Adversarial Imitation Learningcited by 1,577
  4. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 767
  5. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 494
  6. 2018Maximum a Posteriori Policy Optimisationcited by 320
  7. 2016Cooperative Inverse Reinforcement Learningcited by 299
  8. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  9. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 247
  10. 2017DART: Noise Injection for Robust Imitation Learningcited by 188
  11. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  12. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 116
8 more
  1. 2019Leveraging Human Guidance for Deep Reinforcement Learning Taskscited by 42
  2. 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
  3. 2017Pragmatic-Pedagogic Value Alignmentcited by 36
  4. 2019Positive-Unlabeled Reward Learningcited by 36
  5. 2014Interactive Policy Learning through Confidence-Based Autonomycited by 29
  6. 2018Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applicationscited by 28
  7. 2016Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrationscited by 17
  8. 2020Fighting Failures with FIRE: Failure Identification to Reduce Expert Burden in Intervention-Based Learningcited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 5; 2022: 10; 2023: 18; 2024: 20; 2025: 23; 2026: 30.

106 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,197
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 704
  3. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 397
  4. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 313
  5. 2024RT-H: Action Hierarchies Using Languagecited by 207
  6. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  7. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  8. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 137
  9. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
  10. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 118
  11. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 106
  12. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 103
38 more
  1. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 101
  2. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  3. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  4. 2023Data Quality in Imitation Learningcited by 72
  5. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  6. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 67
  7. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 64
  8. 2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
  9. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 57
  10. 2023HYDRA: Hybrid Robot Actions for Imitation Learningcited by 54
  11. 2021LazyDAgger: Reducing Context Switching in Interactive Imitation Learningcited by 50
  12. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  13. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 49
  14. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 44
  15. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 38
  16. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 38
  17. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 35
  18. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Drivingcited by 31
  19. 2022Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervisioncited by 30
  20. 2023Interactive Robot Learning from Verbal Correctioncited by 29
  21. 2023Human-in-the-Loop Task and Motion Planning for Imitation Learningcited by 28
  22. 2025Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Correctionscited by 25
  23. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  24. 2025RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulationcited by 23
  25. 2026NitroGen: An Open Foundation Model for Generalist Gaming Agentscited by 23
  26. 2022Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learningcited by 22
  27. 2022SEIL: Simulation-augmented Equivariant Imitation Learningcited by 22
  28. 2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 22
  29. 2025Learning from Active Human Involvement through Proxy Value Propagationcited by 16
  30. 2026InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoningcited by 14
  31. 2025Human-assisted Robotic Policy Refinement via Action Preference Optimizationcited by 13
  32. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 12
  33. 2022Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based Reinforcement Learningcited by 10
  34. 2023Guarded Policy Optimization with Imperfect Online Demonstrationscited by 10
  35. 2023Residual Q-Learning: Offline and Online Policy Customization without Valuecited by 9
  36. 2023OpenVR: Teleoperation for Manipulationcited by 8
  37. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  38. 2025MILE: Model-based Intervention Learningcited by 8

50 most cited of 106 All 106 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  2. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperation
  3. 2022Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
  4. 2026Instrumentation for Imitation Learning: Enhancing Training Datasets for Clothes Hanger Insertion
  5. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation
  6. 2026PRISM: Personalized Refinement of Imitation Skills for Manipulation via Human Instructions
  7. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
  8. 2026EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policies