mlsanityData to 14 Oct 2026

Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraints

arXiv 2304.08743 ↗First posted 18 Apr 2023cs.LG cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

This study presents a benchmark for evaluating action-constrained reinforcement learning (RL) algorithms. In action-constrained RL, each action taken by the learning system must comply with certain constraints. These constraints are crucial for ensuring the feasibility and safety of actions in real-world systems. We evaluate existing algorithms and their novel variants across multiple robotics control environments, encompassing multiple action constraint types. Our evaluation provides the first in-depth perspective of the field, revealing surprising insights, including the effectiveness of a straightforward baseline approach. The benchmark problems and associated code utilized in our experiments are made available online at github.com/omron-sinicx/action-constrained-RL-benchmark for further research and development.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

20 of 31 references.

References · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2016OpenAI Gymcited by 1,736
  6. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  7. 2019Differentiable Convex Optimization Layerscited by 465
  8. 2019Control Barrier Functions: Theory and Applicationscited by 372
  9. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  10. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  11. 2018Safe Exploration in Continuous Action Spacescited by 247
  12. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
8 more
  1. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  2. 2021Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimizationcited by 31
  3. 2018Resource Constrained Deep Reinforcement Learningcited by 24
  4. 2020Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEscited by 22
  5. 2018Clipped Action Policy Gradientcited by 19
  6. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  7. 2019Solving Online Threat Screening Games using Constrained Action Space Reinforcement Learningcited by 10
  8. 2020Sample-Specific Output Constraints for Neural Networkscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 6; 2026: 2.

11 papers.

  1. 2025Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPscited by 9
  2. 2025Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?cited by 5
  3. 2023DFWLayer: Differentiable Frank-Wolfe Optimization Layercited by 2
  4. 2025Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learningcited by 2
  5. 2025Action-Constrained Imitation Learningcited by 2
  6. 2025Improving Stochastic Action-Constrained Reinforcement Learning via Truncated Distributionscited by 2
  7. 2024Adaptive Safe Reinforcement Learning-Enabled Optimization of Battery Fast-Charging Protocolscited by 1
  8. 2026Differentiable Policy Transport over Multi-Layer Network Feasibility Geometrycited by 1
  9. 2024Safe Reinforcement Learning with Learned Non-Markovian Safety Constraintscited by 0
  10. 2025Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planningcited by 0
  11. 2026Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashingcited by 0