mlsanityData to 7 Oct 2026

OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World

arXiv 1709.07643 ↗First posted 22 Sep 2017cs.RO cs.AIcited by 64

Not on the map. Why?

Abstract (arXiv)

While deep reinforcement learning techniques have recently produced considerable achievements on many decision-making problems, their use in robotics has largely been limited to simulated worlds or restricted motions, since unconstrained trial-and-error interactions in the real world can have undesirable consequences for the robot or its environment. To overcome such limitations, we propose a novel reinforcement learning architecture, OptLayer, that takes as inputs possibly unsafe actions predicted by a neural network and outputs the closest actions that satisfy chosen constraints. While learning control policies often requires carefully crafted rewards and penalties while exploring the range of possible actions, OptLayer ensures that only safe actions are actually executed and unsafe predictions are penalized during training. We demonstrate the effectiveness of our approach on robot reaching tasks, both simulated and in the real world.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 8; 2023: 13; 2024: 13; 2025: 7; 2026: 8.

64 papers.

Sort:

By area:

  1. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  2. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  3. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  4. 2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 41
  5. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  6. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  7. 2021Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimizationcited by 31
  8. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
  9. 2021Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safetycited by 30
  10. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 28
  11. 2021Safe Pontryagin Differentiable Programmingcited by 25
  12. 2021Density Constrained Reinforcement Learningcited by 25
38 more
  1. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  2. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applicationscited by 17
  3. 2022Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environmentscited by 16
  4. 2024FlowPG: Action-constrained Policy Gradient with Normalizing Flowscited by 16
  5. 2022Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interactioncited by 15
  6. 2021Lyapunov Barrier Policy Optimizationcited by 12
  7. 2024Safe Reinforcement Learning on the Constraint Manifold: Theory and Applicationscited by 12
  8. 2024Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agentscited by 11
  9. 2023Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraintscited by 10
  10. 2023Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learningcited by 10
  11. 2023Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functionscited by 9
  12. 2025Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPscited by 8
  13. 2023Quasi-optimal Reinforcement Learning with Continuous Actionscited by 7
  14. 2023Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approachcited by 7
  15. 2021A Closer Look at Advantage-Filtered Behavioral Cloning in High-Noise Datasetscited by 5
  16. 2021Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Interventioncited by 5
  17. 2022Safe reinforcement learning for multi-energy management systems with known constraint functionscited by 5
  18. 2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 5
  19. 2024Efficient Motion Planning for Manipulators with Control Barrier Function-Induced Neural Controllercited by 5
  20. 2025Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?cited by 5
  21. 2021Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learningcited by 4
  22. 2024Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learningcited by 4
  23. 2024Implicit Safe Set Algorithm for Provably Safe Reinforcement Learningcited by 3
  24. 2024A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safetycited by 3
  25. 2021Learning Collision-free and Torque-limited Robot Trajectories based on Alternative Safe Behaviorscited by 2
  26. 2025Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learningcited by 2
  27. 2025Action-Constrained Imitation Learningcited by 2
  28. 2026Improving Feasibility via Fast Autoencoder-Based Projectionscited by 2
  29. 2023An adaptive safety layer with hard constraints for safe reinforcement learning in multi-energy management systemscited by 1
  30. 2024Towards Interpretable Reinforcement Learning with Constrained Normalizing Flow Policiescited by 1
  31. 2024Deep reinforcement learning for weakly coupled MDP's with continuous actionscited by 1
  32. 2026Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Surveycited by 1
  33. 2021Towards a Sample Efficient Reinforcement Learning Pipeline for Vision Based Roboticscited by 0
  34. 2021Monolithic vs. hybrid controller for multi-objective Sim-to-Real learningcited by 0
  35. 2021Transferring Reinforcement Learning for DC-DC Buck Converter Control via Duty Ratio Mapping: From Simulation to Implementationcited by 0
  36. 2022Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programmingcited by 0
  37. 2023Probabilistic Counterexample Guidance for Safer Reinforcement Learning (Extended Version)cited by 0
  38. 2023Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimizationcited by 0

50 most cited of 64 All 64 (JSON)