Multi-skill Mobile Manipulation for Object Rearrangement
Not on the map. Why?
Abstract (arXiv)
We study a modular approach to tackle long-horizon mobile manipulation tasks for object rearrangement, which decomposes a full task into a sequence of subtasks. To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks. Although more effective than monolithic end-to-end RL policies, this framework suffers from compounding errors in skill chaining, e.g., navigating to a bad location where a stationary manipulation skill can not reach its target to manipulate. To this end, we propose that the manipulation skills should include mobility to have flexibility in interacting with the target object from multiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation. We operationalize these ideas by implementing mobile manipulation skills rather than stationary ones and training a navigation skill trained with region goal instead of point goal. We evaluate our multi-skill mobile manipulation method M3 on 3 challenging long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), and show superior performance as compared to the baselines.
Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 6 Sep 2022.
Builds on
20 of 32 references.
References · 20
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
- 2020Learning to Explore using Active Neural SLAMcited by 308
- 2020Integrated Task and Motion Planningcited by 281
- 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
- 2020Rearrangement: A Challenge for Embodied AIcited by 153
- 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 147
8 more
- 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
- 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 89
- 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
- 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
- 2021Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulationcited by 30
- 2019DoorGym: A Scalable Door Opening Environment And Baseline Agentcited by 26
- 2021Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigationcited by 21
- 2021Towards Disturbance-Free Visual Mobile Manipulationcited by 6
Built on by
Citing papers per year: 2022: 2; 2023: 13; 2024: 19; 2025: 6; 2026: 6.
46 papers.
Sort:
By area:
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
- 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
- 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
- 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
- 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
- 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learningcited by 56
- 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
- 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
- 2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4Vcited by 40
- 2025MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulationcited by 34
34 more
- 2024Learning Generalizable Feature Fields for Mobile Manipulationcited by 31
- 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 26
- 2023GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusioncited by 21
- 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 20
- 2024ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Taskscited by 20
- 2023Chain-of-Thought Predictive Controlcited by 18
- 2023Harmonic Mobile Manipulationcited by 16
- 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 15
- 2023Adaptive Coordination in Social Embodied Rearrangementcited by 14
- 2023Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulationcited by 12
- 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 9
- 2023A Closer Look at Reward Decomposition for High-Level Robotic Explanationscited by 7
- 2024When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?cited by 7
- 2024WildLMa: Long Horizon Loco-Manipulation in the Wildcited by 6
- 2022Learning-Augmented Model-Based Planning for Visual Explorationcited by 5
- 2024M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenescited by 5
- 2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Secondcited by 4
- 2025A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewardscited by 4
- 2025ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Makingcited by 4
- 2024Combining Planning and Diffusion for Mobility with Unknown Dynamicscited by 3
- 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrationscited by 3
- 2022Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigationcited by 2
- 2024Reinforcement Learning via Auxiliary Task Distillationcited by 2
- 2025EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environmentscited by 2
- 2026From Instruction to Event: Sound-Triggered Mobile Manipulationcited by 2
- 2026SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMscited by 2
- 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Modelscited by 1
- 2024Hierarchical Object-Oriented POMDP Planning for Object Rearrangementcited by 1
- 2025RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Taskscited by 1
- 2024Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policycited by 0
- 2026Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllerscited by 0
- 2026Diffusion Policy for Coordinated Control of a Nonholonomic Mobile Base and Dual Arms in Door Opening and Passingcited by 0
- 2026MAVP: Map-Aware Visuomotor Policies for Mobile Manipulationcited by 0
- 2026SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptationcited by 0
- 2026SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptation26 Sep 2026
- 2026MAVP: Map-Aware Visuomotor Policies for Mobile Manipulation22 Sep 2026
- 2026Diffusion Policy for Coordinated Control of a Nonholonomic Mobile Base and Dual Arms in Door Opening and Passing14 May 2026
- 2026Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers14 Apr 2026
- 2026SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs26 Feb 2026
- 2026From Instruction to Event: Sound-Triggered Mobile Manipulation29 Jan 2026
- 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations24 Aug 2025
- 2025RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks7 Jun 2025
- 2025ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making27 May 2025
- 2025MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulation17 Mar 2025
- 2025EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environments11 Mar 2025
- 2025A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards12 Feb 2025
34 more
- 2024When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?18 Dec 2024
- 2024ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks9 Dec 2024
- 2024Hierarchical Object-Oriented POMDP Planning for Object Rearrangement2 Dec 2024
- 2024WildLMa: Long Horizon Loco-Manipulation in the Wild22 Nov 2024
- 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes15 Oct 2024
- 2024Combining Planning and Diffusion for Mobility with Unknown Dynamics9 Oct 2024
- 2024M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes9 Oct 2024
- 2024Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy8 Oct 2024
- 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RL30 Sep 2024
- 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Models28 Aug 2024
- 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learning29 Jul 2024
- 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators28 Jun 2024
- 2024Reinforcement Learning via Auxiliary Task Distillation24 Jun 2024
- 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction16 May 2024
- 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks25 Apr 2024
- 2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V16 Apr 2024
- 2024Visual Whole-Body Control for Legged Loco-Manipulation25 Mar 2024
- 2024Learning Generalizable Feature Fields for Mobile Manipulation12 Mar 2024
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan 2024
- 2023Harmonic Mobile Manipulation11 Dec 2023
- 2023Large Language Models as Generalizable Policies for Embodied Tasks26 Oct 2023
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots19 Oct 2023
- 2023GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion27 Sep 2023
- 2023Skill Transformer: A Monolithic Policy for Mobile Manipulation19 Aug 2023
- 2023Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation12 Jul 2023
- 2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second13 Jun 2023
- 2023Adaptive Coordination in Social Embodied Rearrangement31 May 2023
- 2023A Closer Look at Reward Decomposition for High-Level Robotic Explanations25 Apr 2023
- 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes9 Apr 2023
- 2023Chain-of-Thought Predictive Control3 Apr 2023
- 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?31 Mar 2023
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills9 Feb 2023
- 2022Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigation17 Dec 2022
- 2022Learning-Augmented Model-Based Planning for Visual Exploration15 Nov 2022
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
- 2024WildLMa: Long Horizon Loco-Manipulation in the Wildcited by 6
- 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Modelscited by 1
- 2026Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllerscited by 0
- 2026Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers14 Apr 2026
- 2024WildLMa: Long Horizon Loco-Manipulation in the Wild22 Nov 2024
- 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Models28 Aug 2024
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan 2024
- 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 26
- 2024Combining Planning and Diffusion for Mobility with Unknown Dynamicscited by 3
- 2026Diffusion Policy for Coordinated Control of a Nonholonomic Mobile Base and Dual Arms in Door Opening and Passingcited by 0
- 2026Diffusion Policy for Coordinated Control of a Nonholonomic Mobile Base and Dual Arms in Door Opening and Passing14 May 2026
- 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes15 Oct 2024
- 2024Combining Planning and Diffusion for Mobility with Unknown Dynamics9 Oct 2024
- 2023Chain-of-Thought Predictive Controlcited by 18
- 2023Chain-of-Thought Predictive Control3 Apr 2023
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
- 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
- 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
- 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
- 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
- 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learningcited by 56
- 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
- 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
- 2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4Vcited by 40
- 2024Learning Generalizable Feature Fields for Mobile Manipulationcited by 31
- 2023GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusioncited by 21
12 most cited of 33 All 33 (JSON)
- 2026MAVP: Map-Aware Visuomotor Policies for Mobile Manipulation22 Sep 2026
- 2026From Instruction to Event: Sound-Triggered Mobile Manipulation29 Jan 2026
- 2025RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks7 Jun 2025
- 2025EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environments11 Mar 2025
- 2025A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards12 Feb 2025
- 2024When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?18 Dec 2024
- 2024ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks9 Dec 2024
- 2024Hierarchical Object-Oriented POMDP Planning for Object Rearrangement2 Dec 2024
- 2024M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes9 Oct 2024
- 2024Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy8 Oct 2024
- 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RL30 Sep 2024
- 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learning29 Jul 2024
12 newest of 33 All 33 (JSON)