# Robot Learning of Mobile Manipulation with Reachability Behavior Priors

arXiv: https://arxiv.org/abs/2203.04051 · HTML: https://mlsanity.com/paper/2203.04051/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives](https://mlsanity.com/paper/2110.15360/) (2021)
- [HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators](https://mlsanity.com/paper/1910.11432/) (2019)
- [Information asymmetry in KL-regularized RL](https://mlsanity.com/paper/1905.01240/) (2019)
- [MushroomRL: Simplifying Reinforcement Learning Research](https://mlsanity.com/paper/2001.01102/) (2020)
- [ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation](https://mlsanity.com/paper/2008.07792/) (2020)
- [Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics](https://mlsanity.com/paper/2001.00449/) (2020)
- [Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2103.10534/) (2021)
- [Control Regularization for Reduced Variance Reinforcement Learning](https://mlsanity.com/paper/1905.05380/) (2019)
- [Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulation](https://mlsanity.com/paper/2107.13545/) (2021)
- [Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.13842/) (2021)
- [Learning Kinematic Feasibility for Mobile Manipulation through Deep Reinforcement Learning](https://mlsanity.com/paper/2101.05325/) (2021)
- [Learning Dexterous Manipulation from Suboptimal Experts](https://mlsanity.com/paper/2010.08587/) (2020)
- [Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Robotics](https://mlsanity.com/paper/2107.09822/) (2021)
- [Reuleaux: Robot Base Placement by Reachability Analysis](https://mlsanity.com/paper/1710.01328/) (2017)
- [Whole-Body Control of a Mobile Manipulator using End-to-End Reinforcement Learning](https://mlsanity.com/paper/2003.02637/) (2020)
- [Combined Task and Action Learning from Human Demonstrations for Mobile Manipulation Applications](https://mlsanity.com/paper/1908.10184/) (2019)

## Built on by (most cited first, 22 of 22)

- [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient](https://mlsanity.com/paper/2406.00681/) (2024)
- [GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion](https://mlsanity.com/paper/2309.15459/) (2023)
- [Causal Policy Gradient for Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2305.04866/) (2023)
- [Harmonic Mobile Manipulation](https://mlsanity.com/paper/2312.06639/) (2023)
- [N2M2: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments](https://mlsanity.com/paper/2206.08737/) (2022)
- [Residual Q-Learning: Offline and Online Policy Customization without Value](https://mlsanity.com/paper/2306.09526/) (2023)
- [Active-Perceptive Motion Generation for Mobile Manipulation](https://mlsanity.com/paper/2310.00433/) (2023)
- [Mobi-π: Mobilizing Your Robot Learning Policy](https://mlsanity.com/paper/2505.23692/) (2025)
- [Hierarchical Policy Blending as Inference for Reactive Robot Control](https://mlsanity.com/paper/2210.07890/) (2022)
- [SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Training](https://mlsanity.com/paper/2506.04147/) (2025)
- [Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering](https://mlsanity.com/paper/2306.07392/) (2023)
- [Hierarchical Policy Blending As Optimal Transport](https://mlsanity.com/paper/2212.01938/) (2022)
- [Planning Robot Placement for Object Grasping](https://mlsanity.com/paper/2405.16692/) (2024)
- [Deep Reinforcement Learning for Reach-Avoid-Stay Problems](https://mlsanity.com/paper/2410.02898/) (2024)
- [QuadWBG: Generalizable Quadrupedal Whole-Body Grasping](https://mlsanity.com/paper/2411.06782/) (2024)
- [A Reactive Framework for Whole-Body Motion Planning of Mobile Manipulators Combining Reinforcement Learning and SDF-Constrained Quadratic Programmi](https://mlsanity.com/paper/2503.23975/) (2025)
- [Human Centric General Physical Intelligence for Agile Manufacturing Automation](https://mlsanity.com/paper/2508.11960/) (2025)
- [Manipulate-to-Navigate: Reinforcement Learning with Visual Affordances and Manipulability Priors](https://mlsanity.com/paper/2508.13151/) (2025)
- [Predictive Reachability for Embodiment Selection in Mobile Manipulation Behaviors](https://mlsanity.com/paper/2410.21059/) (2024)
- [Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers](https://mlsanity.com/paper/2604.12509/) (2026)

## Abstract (arXiv)

Mobile Manipulation (MM) systems are ideal candidates for taking up the role of a personal assistant in unstructured real-world environments. Among other challenges, MM requires effective coordination of the robot's embodiments for executing tasks that require both mobility and manipulation. Reinforcement Learning (RL) holds the promise of endowing robots with adaptive behaviors, but most methods require prohibitively large amounts of data for learning a useful control policy. In this work, we study the integration of robotic reachability priors in actor-critic RL methods for accelerating the learning of MM for reaching and fetching tasks. Namely, we consider the problem of optimal base placement and the subsequent decision of whether to activate the arm for reaching a 6D target. For this, we devise a novel Hybrid RL method that handles discrete and continuous actions jointly, resorting to the Gumbel-Softmax reparameterization. Next, we train a reachability prior using data from the operational robot workspace, inspired by classical methods. Subsequently, we derive Boosted Hybrid RL (BHyRL), a novel algorithm for learning Q-functions by modeling them as a sum of residual approximators. Every time a new task needs to be learned, we can transfer our learned residuals and learn the component of the Q-function that is task-specific, hence, maintaining the task structure from prior behaviors. Moreover, we find that regularizing the target policy with a prior policy yields more expressive behaviors. We evaluate our method in simulation in reaching and fetching tasks of increasing difficulty, and we show the superior performance of BHyRL against baseline methods. Finally, we zero-transfer our learned 6D fetching policy with BHyRL to our MM robot TIAGo++. For more details and code release, please refer to our project site: irosalab.com/rlmmbp