mlsanityData to 14 Oct 2026

Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance

arXiv 2610.20607 ↗First posted 14 Oct 2026cs.RO cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Mobile Manipulation (MoMa) is a cornerstone of autonomous robotic systems, yet real-world fine-tuning remains a formidable challenge because of high-dimensional action spaces and intricate coupling between heterogeneous modules. In this paper, we propose HiRe-MoMa (Hierarchical Residual Mobile Manipulation), a systematic framework for stable and sample-efficient real-world fine-tuning of complex MoMa systems. At its core, HiRe-MoMa adopts a Hierarchical Residual Architecture that fine-tunes a frozen base policy. A Policy-Agnostic Residual learns additive corrections atop the base policy, while Hierarchical Decoupling splits the residual into mobile-base and manipulator-arm heads with unidirectional conditioning, mitigating gradient interference and aligning corrections with the platform's physical hierarchy. To improve sample efficiency in data-expensive real-world settings, we propose Expert-Guided Value Alignment, which leverages sparse human interventions with under 20% training coverage to reshape the critic's value landscape, effectively guiding autonomous exploration through expert priors. Extensive real-world experiments on the TIAGo++ platform demonstrate that HiRe-MoMa consistently enhances task performance with only around 30 minutes of online interaction, regardless of the base policy architecture. To our knowledge, this is among the first frameworks to enable efficient on-robot fine-tuning for mobile manipulation in the real world, providing a practical paradigm for deploying complex robotic systems in dynamic, human-centric environments. Further details can be found on our project website: https://hire-moma.github.io/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 59 of its 66 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

59 of 66 references.

References · 59

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  6. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  7. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  8. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  9. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
  10. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 189
  11. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  12. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 154
38 more
  1. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 136
  2. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 108
  3. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
  4. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learningcited by 97
  5. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 94
  6. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 90
  7. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 84
  8. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 79
  9. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
  10. 2025Efficient Online Reinforcement Learning for Diffusion Policycited by 71
  11. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 63
  12. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 60
  13. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  14. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 55
  15. 2025VLAS: Vision-Language-Action Model With Speech Instructions For Customized Robot Manipulationcited by 51
  16. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 49
  17. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 48
  18. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 47
  19. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 39
  20. 2024Inference-Time Policy Steering through Human Interactionscited by 38
  21. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 34
  22. 2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulationcited by 32
  23. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  24. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  25. 2024CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Predictioncited by 27
  26. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 23
  27. 2025AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulationcited by 23
  28. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  29. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  30. 2023Causal Policy Gradient for Whole-Body Mobile Manipulationcited by 21
  31. 2024Catch It! Learning to Catch in Flight with Mobile Dexterous Handscited by 16
  32. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 15
  33. 2025MILE: Model-based Intervention Learningcited by 10
  34. 2025GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learningcited by 9
  35. 2025Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarmcited by 6
  36. 2025SafeMimic: Towards Safe and Autonomous Human-to-Robot Imitation for Mobile Manipulationcited by 6
  37. 2024Robust Offline Imitation Learning from Diverse Auxiliary Datacited by 5
  38. 2025SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-trainingcited by 5

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.