# Force-Aware Reinforcement Learning with Hybrid Sensorless Force Estimation for Wheeled-Legged Loco-Manipulation

arXiv: https://arxiv.org/abs/2609.13779 · HTML: https://mlsanity.com/paper/2609.13779/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 12 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 23 of 23)

- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10602/) (2023)
- [Visual Whole-Body Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2403.16967/) (2024)
- [Adapting Rapid Motor Adaptation for Bipedal Robots](https://mlsanity.com/paper/2205.15299/) (2022)
- [Combining Learning-based Locomotion Policy with Model-based Manipulation for Legged Mobile Manipulators](https://mlsanity.com/paper/2201.03871/) (2022)
- [Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning](https://mlsanity.com/paper/2310.10509/) (2023)
- [Learning Force Control for Legged Manipulation](https://mlsanity.com/paper/2405.01402/) (2024)
- [RoLoMa: Robust Loco-Manipulation for Quadruped Robots with Arms](https://mlsanity.com/paper/2203.01446/) (2022)
- [FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robots](https://mlsanity.com/paper/2505.06883/) (2025)
- [Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator](https://mlsanity.com/paper/2412.03012/) (2024)
- [Whole-body End-Effector Pose Tracking](https://mlsanity.com/paper/2409.16048/) (2024)
- [Contact Optimization for Non-Prehensile Loco-Manipulation via Hierarchical Model Predictive Control](https://mlsanity.com/paper/2210.03442/) (2022)
- [Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation](https://mlsanity.com/paper/2505.20829/) (2025)
- [Proprioceptive External Torque Learning for Floating Base Robot and its Applications to Humanoid Locomotion](https://mlsanity.com/paper/2309.04138/) (2023)
- [Efficient Learning of A Unified Policy For Whole-body Manipulation and Locomotion Skills](https://mlsanity.com/paper/2507.04229/) (2025)
- [Fine Robotic Manipulation without Force/Torque Sensor](https://mlsanity.com/paper/2301.13413/) (2023)
- [Sensorless Estimation of Contact Using Deep-Learning for Human-Robot Interaction](https://mlsanity.com/paper/2309.16219/) (2023)
- [Collision detection and identification for a legged manipulator](https://mlsanity.com/paper/2207.14745/) (2022)
- [Multi-Momentum Observer Contact Estimation for Bipedal Robots](https://mlsanity.com/paper/2412.03462/) (2024)

## Abstract (arXiv)

Force-controlled loco-manipulation requires a whole-body policy to coordinate locomotion and arm motion while regulating end-effector interaction forces. This is challenging under floating-base dynamics and changing support contacts, particularly when end-effector force/torque sensing is unavailable for control. This paper presents a force-aware reinforcement learning approach with hybrid sensorless force estimation for wheeled-legged loco-manipulation. The proposed method provides a structured estimate of the end-effector force as an explicit policy observation, enabling force-guided contact behavior without using an end-effector force/torque sensor for control. The force estimate is obtained by combining generalized momentum observation, contact-constrained wrench projection, and temporal residual learning: the model-based components extract the physically structured part of the whole-body disturbance, while the residual network compensates the remaining motion-dependent bias. The estimated force is integrated into a mode-conditioned whole-body policy with an axis-wise force/position selector, allowing free-space motion, pure force regulation, and hybrid force/position control within one controller. Simulation results demonstrate improved sensorless force estimation and force-control performance. Hardware experiments further validate the proposed controller through quantitative valve-rotation and hybrid wiping evaluations, together with force-guided door opening and zero-force human-guided motion on a real wheeled-legged platform.