mlsanityData to 7 Oct 2026

Robust Imitation Learning for Mobile Manipulator Focusing on Task-Related Viewpoints and Regions

arXiv 2410.01292 ↗First posted 2 Oct 2024cs.ROcited by 2

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

We study how to generalize the visuomotor policy of a mobile manipulator from the perspective of visual observations. The mobile manipulator is prone to occlusion owing to its own body when only a single viewpoint is employed and a significant domain shift when deployed in diverse situations. However, to the best of the authors' knowledge, no study has been able to solve occlusion and domain shift simultaneously and propose a robust policy. In this paper, we propose a robust imitation learning method for mobile manipulators that focuses on task-related viewpoints and their spatial regions when observing multiple viewpoints. The multiple viewpoint policy includes attention mechanism, which is learned with an augmented dataset, and brings optimal viewpoints and robust visual embedding against occlusion and domain shift. Comparison of our results for different tasks and environments with those of previous studies revealed that our proposed method improves the success rate by up to 29.3 points. We also conduct ablation studies using our proposed method. Learning task-related viewpoints from the multiple viewpoints dataset increases robustness to occlusion than using a uniquely defined viewpoint. Focusing on task-related regions contributes to up to a 33.3-point improvement in the success rate against domain shift.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 31 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

18 of 31 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan
  2. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  3. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb

Other references · 15

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  6. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  7. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  8. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  9. 2023Fast Segment Anythingcited by 286
  10. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  11. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  12. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
3 more
  1. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  2. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  3. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Object-Centric Mobile Manipulation through SAM2-Guided Perception and Imitation Learningcited by 1
  2. 2026Task-Relevant and Irrelevant Region-Aware Augmentation for Generalizable Vision-Based Imitation Learning in Agricultural Manipulationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2021Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations
  2. 2021Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation
  3. 2025M4Diffuser: Multi-View Diffusion Policy with Manipulability-Aware Control for Robust Mobile Manipulation
  4. 2026An Empirical Study on What Matters for Viewpoint-Generalizable Policies in Visual Imitation Learning
  5. 2025Seeing the Bigger Picture: 3D Latent Mapping for Mobile Manipulation Policy Learning
  6. 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Models
  7. 2026Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Camera
  8. 2026MobileManiBench: Simplifying Model Verification for Mobile Manipulation