mlsanityData to 7 Oct 2026

Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation

arXiv 2112.05251 ↗First posted 9 Dec 2021cs.RO cs.AI cs.LGcited by 59

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

In mobile manipulation (MM), robots can both navigate within and interact with their environment and are thus able to complete many more tasks than robots only capable of navigation or manipulation. In this work, we explore how to apply imitation learning (IL) to learn continuous visuo-motor policies for MM tasks. Much prior work has shown that IL can train visuo-motor policies for either manipulation or navigation domains, but few works have applied IL to the MM domain. Doing this is challenging for two reasons: on the data side, current interfaces make collecting high-quality human demonstrations difficult, and on the learning side, policies trained on limited data can suffer from covariate shift when deployed. To address these problems, we first propose Mobile Manipulation RoboTurk (MoMaRT), a novel teleoperation framework allowing simultaneous navigation and manipulation of mobile manipulators, and collect a first-of-its-kind large scale dataset in a realistic simulated kitchen setting. We then propose a learned error detection system to address the covariate shift by detecting when an agent is in a potential failure state. We train performant IL policies and error detectors from this data, and achieve over 45% task success rate and 85% error detection success rate across multiple multi-stage tasks when trained on expert data. Codebase, datasets, visualization, and more available at https://sites.google.com/view/il-for-mm/home.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 70 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

25 of 70 references.

Closest ancestors · 5

Same direction, newest first

  1. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperation12 Dec
  2. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  3. 2019Combined Task and Action Learning from Human Demonstrations for Mobile Manipulation Applications25 Aug
  4. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov
  5. 2018Shared Multi-Task Imitation Learning for Indoor Self-Navigation14 Aug

Other references · 20

  1. 2013Auto-Encoding Variational Bayescited by 3,980
  2. 2019Deep Learning for Anomaly Detection: A Surveycited by 357
  3. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  4. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  5. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  6. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 96
  7. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 89
  8. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidancecited by 86
  9. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 52
  10. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 47
  11. 2021Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulationcited by 44
  12. 2016FFRob: Leveraging Symbolic Planning for Efficient Task and Motion Planningcited by 34
8 more
  1. 2019VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learningcited by 29
  2. 2018GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimationcited by 28
  3. 2021VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigationcited by 27
  4. 2021Learning Kinematic Feasibility for Mobile Manipulation through Deep Reinforcement Learningcited by 24
  5. 2020Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensorscited by 17
  6. 2018Failure Prediction for Autonomous Drivingcited by 14
  7. 2019The RobotriX: An eXtremely Photorealistic and Very-Large-Scale Indoor Dataset of Sequences with Robot Trajectories and Interactionscited by 14
  8. 2019Whole-Body MPC for a Dynamically Stable Mobile Manipulatorcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 20; 2025: 14; 2026: 12.

59 papers.

Sort:

By area:

  1. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  2. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  3. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  4. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  5. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  6. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  7. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 74
  8. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
  9. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
  10. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 49
  11. 2025Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policiescited by 49
  12. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
38 more
  1. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  2. 2025Failure Prediction at Runtime for Generative Robot Policiescited by 36
  3. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 33
  4. 2024ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperationcited by 32
  5. 2024DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulationcited by 32
  6. 2024Learning Generalizable Feature Fields for Mobile Manipulationcited by 31
  7. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 30
  8. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  9. 2024Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulationcited by 29
  10. 2023Interactive Robot Learning from Verbal Correctioncited by 28
  11. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 27
  12. 2024SPIN: Simultaneous Perception, Interaction and Navigationcited by 27
  13. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 26
  14. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  15. 2024TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learningcited by 21
  16. 2025AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulationcited by 21
  17. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
  18. 2022N2M2: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environmentscited by 15
  19. 2023Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximationcited by 15
  20. 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 15
  21. 2025Responsive Noise-Relaying Diffusion Policy: Responsive and Efficient Visuomotor Controlcited by 13
  22. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 13
  23. 2025MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulationcited by 12
  24. 2023MOMA-Force: Visual-Force Imitation for Real-World Mobile Manipulationcited by 9
  25. 2025Tool-as-Interface: Learning Robot Policies from Observing Human Tool Usecited by 8
  26. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
  27. 2024Whole-Body Teleoperation for Mobile Manipulation at Zero Added Costcited by 6
  28. 2024WildLMa: Long Horizon Loco-Manipulation in the Wildcited by 6
  29. 2026VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memorycited by 6
  30. 2025SafeMimic: Towards Safe and Autonomous Human-to-Robot Imitation for Mobile Manipulationcited by 5
  31. 2024λ: A Benchmark for Data-Efficiency in Long-Horizon Indoor Mobile Manipulation Roboticscited by 3
  32. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environmentscited by 3
  33. 2026SAFECAST: Robust Failure Detection for VLA Policies with Contrast-Set Training and Calibrationcited by 2
  34. 2026AutoIntervene: Calibrated Intervention for Action-Chunking Imitation Learning Policiescited by 2
  35. 2025Feasibility-aware Imitation Learning from Observations through a Hand-mounted Demonstration Interfacecited by 1
  36. 2026Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Modelscited by 1
  37. 2023How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushingcited by 0
  38. 2024Know your limits! Optimize the robot's behavior through self-awarenesscited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Models
  2. 2025EMMA: Scaling Mobile Manipulation via Egocentric Human Data
  3. 2025Object-Centric Mobile Manipulation through SAM2-Guided Perception and Imitation Learning
  4. 2026Beyond the Majority: Long-tail Imitation Learning for Robotic Manipulation
  5. 2024Robust Imitation Learning for Mobile Manipulator Focusing on Task-Related Viewpoints and Regions
  6. 2026MobileManiBench: Simplifying Model Verification for Mobile Manipulation
  7. 2025Online Imitation Learning for Manipulation via Decaying Relative Correction through Teleoperation
  8. 2026WM-DAgger: Enabling Efficient Data Aggregation for Imitation Learning with World Models