mlsanityData to 14 Oct 2026

Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing

arXiv 2605.04185 ↗First posted 5 May 2026cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

When deploying reinforcement learning policies to physical robots, actuator rate constraints -- hard limits on how fast each joint can move per control step -- are unavoidable. These limits vary substantially across joints due to differences in motor inertia, power bandwidth, and transmission stiffness, creating pronounced heterogeneity that existing methods fail to handle geometrically: the per-joint feasible region forms a high-dimensional box in action-increment space, yet QP projection and spherical parameterization methods impose isotropic ball-shaped constraints, exponentially under-covering the true feasible set as heterogeneity grows. This paper proposes Dynamic Decoupled Spherical Radial Squashing (DD-SRad), which resolves this mismatch by computing a position-adaptive radius independently for each actuator, achieving tight alignment with the true per-joint feasible region. DD-SRad satisfies per-step hard constraints with probability~1, preserves well-conditioned gradients throughout training, and admits exact policy gradient backpropagation with zero runtime solver overhead. MuJoCo benchmark experiments demonstrate the highest task return at zero constraint violation -- matching the unconstrained upper bound -- with 30%--50% improvement in constraint-space coverage over spherical baselines. High-fidelity IsaacLab simulations with Unitree H1 and G1 humanoid robots confirm end-to-end optimality parameterized directly from official joint specifications, validating a systematic pathway from hardware datasheets to safe deployment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 38 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

29 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Learning Sim-to-Real Humanoid Locomotion in 15 Minutes1 Dec

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2017Constrained Policy Optimizationcited by 983
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  7. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  8. 2018Reward Constrained Policy Optimizationcited by 417
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  10. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  11. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  12. 2018Residual Reinforcement Learning for Robot Controlcited by 274
16 more
  1. 2018Safe Exploration in Continuous Action Spacescited by 247
  2. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  3. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  4. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 68
  5. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 68
  6. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  7. 2019Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Controlcited by 54
  8. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 44
  9. 2022Perceptive Locomotion through Nonlinear Model Predictive Controlcited by 38
  10. 2021A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learningcited by 18
  11. 2024FlowPG: Action-constrained Policy Gradient with Normalizing Flowscited by 16
  12. 2023Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraintscited by 11
  13. 2025Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPscited by 9
  14. 2020Safety-Critical Kinematic Control of Robotic Systemscited by 7
  15. 2025Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learningcited by 2
  16. 2025Improving Stochastic Action-Constrained Reinforcement Learning via Truncated Distributionscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Physics-Informed Policy Optimization via Analytic Dynamics Regularization
  2. 2026Latent Spherical Flow Policy for Reinforcement Learning with Combinatorial Actions
  3. 2026Actuator Dynamics Curricula for Narrow-Viability Tasks in Legged Robot Learning
  4. 2026Geometric Pareto Control: Physics-Supervised Pareto Representation Learning via Riemannian Energy-Gradient Flow ↗
  5. 2024Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning
  6. 2026Hybrid TD3: Overestimation Bias Analysis and Stable Policy Optimization for Hybrid Action Space
  7. 2026PPO-EAL: Exact Augmented Lagrangian Proximal Policy Optimization for Safe Robotic Control
  8. 2025Prior Reinforce: Goal-Conditioned Dynamic Manipulation with Limited Trials ↗