mlsanityData to 14 Oct 2026

RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulation

arXiv 2503.07771 ↗First posted 10 Mar 2025cs.ROcited by 26

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Learning from human demonstration is an effective approach for learning complex manipulation skills. However, existing approaches heavily focus on learning from passive human demonstration data for its simplicity in data collection. Interactive human teaching has appealing theoretical and practical properties, but they are not well supported by existing human-robot interfaces. This paper proposes a novel system that enables seamless control switching between human and an autonomous policy for bi-manual manipulation tasks, enabling more efficient learning of new tasks. This is achieved through a compliant, bilateral teleoperation system. Through simulation and hardware experiments, we demonstrate the value of our system in an interactive human teaching for learning complex bi-manual manipulation skills.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 35 of its 66 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

35 of 66 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformer31 Jan
  2. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan
  3. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators22 Sep
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  5. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations2 Mar
  6. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors20 Oct
  7. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  8. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperation12 Dec
  9. 2020Visual Imitation Made Easy11 Aug

Other references · 26

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  6. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  7. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  8. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  9. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  10. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
  11. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
  12. 2017DART: Noise Injection for Robust Imitation Learningcited by 191
14 more
  1. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  2. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 108
  3. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  4. 2023On Bringing Robots Homecited by 101
  5. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 84
  6. 2023LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learningcited by 79
  7. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  8. 2022Interactive Imitation Learning in Robotics: A Surveycited by 55
  9. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 49
  10. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  11. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  12. 2021NimbRo Avatar: Interactive Immersive Telepresence with Force-Feedback Telemanipulationcited by 13
  13. 2019Quasi-Direct Drive for Low-Cost Compliant Robotic Manipulationcited by 5
  14. 2021Bimanual Telemanipulation with Force and Haptic Feedback and Predictive Limit Avoidancecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 16.

26 papers.

Sort:

By area:

  1. 2025Failure Prediction at Runtime for Generative Robot Policiescited by 42
  2. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 39
  3. 2026RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Taskscited by 33
  4. 2025Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Correctionscited by 29
  5. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  6. 2026InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoningcited by 14
  7. 2026DeMaVLA: A Vision-Language-Action Foundation Model for Generalizable Deformable Manipulationcited by 9
  8. 2025HACTS: a Human-As-Copilot Teleoperation System for Robot Learningcited by 8
  9. 2025Real-world Reinforcement Learning from Suboptimal Interventionscited by 8
  10. 2026FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulationcited by 8
  11. 2025Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidancecited by 7
  12. 2026ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Trainingcited by 7
14 more
  1. 2025Human-in-the-loop Online Rejection Sampling for Robotic Manipulationcited by 5
  2. 2025Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Surveycited by 4
  3. 2025Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoningcited by 3
  4. 2026RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Modelscited by 2
  5. 2026Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Interventioncited by 2
  6. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulationcited by 2
  7. 2026Robot Self-Improvement via Human-Video Dynamics Modelscited by 2
  8. 2026Human-Robot Copilot for Data-Efficient Imitation Learningcited by 1
  9. 2026One Demonstration Is Enough for Real-World Robotic Reinforcement Learningcited by 1
  10. 2026DexPIE: Stable Dexterous Policy Improvement from Real-World Experiencecited by 0
  11. 2026Policy-Calibrated DAgger: Offline Calibrated Noise Injection for Imitation Learningcited by 0
  12. 2026BlenDAgger: Blended Shared Control for Interactive Imitation Learningcited by 0
  13. 2026Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learningcited by 0
  14. 2026I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies
  2. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
  3. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulation
  4. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
  5. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  6. 2024VITAL: Interactive Few-Shot Imitation Learning via Visual Human-in-the-Loop Corrections
  7. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation
  8. 2024ALPHA-α and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System