mlsanityData to 14 Oct 2026

Stability-aware Residual Reinforcement Learning Framework for Robotic Manipulator Disturbance Compensation

arXiv 2609.21307 ↗First posted 18 Sep 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Although conventional controllers and disturbance observers (DOBs) are the standard for precision tracking in manipulators, they suffer from parameter uncertainty, nonlinear friction, and compound disturbances. This study proposes a residual reinforcement learning DOB framework that pairs an analytical observer with an RL policy. The deterministic baseline operates within a reliable region, whereas the RL policy explicitly targets the residuals that the model cannot capture. To make this compensation disturbance-aware, an estimator network aligns the observation history with a privileged disturbance context, organizing the latent space by disturbance regime and enabling rapid adaptation across disturbance transitions. To guarantee stability, we derived and enforced a state-dependent action bound on the RL policy from an input-to-state stability (ISS) analysis such that the closed loop provably confines the tracking error to a certified envelope for arbitrary policy outputs. Experiments on a 6-DOF manipulator demonstrated consistent improvements in disturbance estimation and tracking, including a 27.8% tracking-error reduction on real hardware under zero-shot sim-to-real transfer and a 38.0% reduction under a base-vibration disturbance that was not observed during training.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 33 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

17 of 33 references.

References · 17

  1. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  2. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  5. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  6. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  7. 2018Residual Policy Learningcited by 164
  8. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
  9. 2023IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Realitycited by 53
  10. 2023Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Responsecited by 39
  11. 2025Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robotscited by 12
  12. 2023DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Controlcited by 7
5 more
  1. 2023Residual Policy Learning for Vehicle Control of Autonomous Racing Carscited by 6
  2. 2019Disturbance Observer-based Robust Control and Its Applications: 35th Anniversary Overviewcited by 4
  3. 2025RL-augmented Adaptive Model Predictive Control for Bipedal Locomotion over Challenging Terraincited by 3
  4. 2025Residual MPC: Blending Reinforcement Learning with GPU-Parallelized Model Predictive Controlcited by 3
  5. 2019DOB-Net: Actively Rejecting Unknown Excessive Time-Varying Disturbancescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tackling Sim-to-Real Mismatch Through Sampling-Based Disturbance Observers: From Analytical Models to Learned World Models
  2. 2025Disturbance-Aware Adaptive Compensation in Hybrid Force-Position Locomotion Policy for Legged Robots
  3. 2024Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning
  4. 2024Disturbance Observer-based Control Barrier Functions with Residual Model Learning for Safe Reinforcement Learning
  5. 2024Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances
  6. 2026Residual Reinforcement Learning for Robot Teleoperation under Stochastic Delays
  7. 2021Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies
  8. 2026Residual Deep Reinforcement Learning-Based Computed Torque Control for a Cable-Driven Lower-Limb Rehabilitation Robot under Disturbances and Parametric Uncertainties