Contraction Metric Based Safe Reinforcement Learning Force Control for a Hydraulic Actuator with Real-World Training
On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control
Abstract (arXiv)
Force control in hydraulic actuators is notoriously difficult due to strong nonlinearities, uncertainties, and the high risks associated with unsafe exploration during learning. This paper investigates safe reinforcement learning (RL) for hy draulic force control with real-world training using contraction metric certificates. A data-driven model of a hydraulic actuator, identified from experimental data, is employed for simulation based pretraining of a Soft Actor-Critic (SAC) policy that adapts the PI gains of a feedback-linearization (FL) controller. To reduce instability during online training, we propose a quadratic-programming (QP) contraction filter that leverages a learned contraction metric to enforce approximate exponential convergence of trajectories, applying minimal corrections to the policy output. The approach is validated on a hydraulic test bench, where the RL controller is trained directly on hardware and benchmarked against a simulation-trained agent and a fixed-gain baseline. Experimental results show that real-hardware training improves force-tracking performance compared to both alternatives, while the contraction filter mitigates chattering and instabilities. These findings suggest that contraction-based certificates can enable safe RL in high force hydraulic systems, though robustness at extreme operating conditions remains a challenge.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 32 references that are in the map. First posted on arXiv 9 Feb 2026.
Builds on
14 of 32 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2017Constrained Policy Optimizationcited by 983
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
- 2017Safe Reinforcement Learning via Shieldingcited by 449
- 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
- 2022Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methodscited by 151
- 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 94
- 2020Learning Certified Control using Contraction Metriccited by 49
- 2022skrl: Modular and Flexible Library for Reinforcement Learningcited by 49
- 2015Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Designcited by 39
- 2025Towards Learning Boulder Excavation with Hydraulic Excavatorscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model
- 2026Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator Control
- 2025Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustness
- 2024Reinforcement Learning Controllers for Soft Robots using Learned Environments
- 2025MPC-Guided Safe Reinforcement Learning and Lipschitz-Based Filtering for Structured Nonlinear Systems
- 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
- 2026ContractionPPO: Certified Reinforcement Learning via Differentiable Contraction Layers
- 2025Autonomous Control of Redundant Hydraulic Manipulator Using Reinforcement Learning with Action Feedback