mlsanityData to 14 Oct 2026

Constrained Reinforcement Learning for Robotics via Scenario-Based Programming

arXiv 2206.09603 ↗First posted 20 Jun 2022cs.RO cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has achieved groundbreaking successes in a wide variety of robotic applications. A natural consequence is the adoption of this paradigm for safety-critical tasks, where human safety and expensive hardware can be involved. In this context, it is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our technique exploits the scenario-based programming paradigm, which is designed to allow specifying such knowledge in a simple and intuitive way. We validated our method on the popular robotic mapless navigation problem, in simulation, and on the actual platform. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 33 of its 76 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

33 of 76 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
  6. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  7. 2017Deep Reinforcement Learning: An Overviewcited by 333
  8. 2016Safety Verification of Deep Neural Networkscited by 263
  9. 2018Unity: A General Platform for Intelligent Agentscited by 236
  10. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 225
  11. 2019Algorithms for Verifying Deep Neural Networkscited by 221
  12. 2018Formal Security Analysis of Neural Networks using Symbolic Intervalscited by 185
21 more
  1. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
  2. 2017Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigationcited by 98
  3. 2016Deep Reinforcement Learning with Successor Features for Navigation across Similar Environmentscited by 50
  4. 2018Reinforcement Learning for UAV Attitude Controlcited by 48
  5. 2020An SMT-Based Approach for Verifying Binarized Neural Networkscited by 47
  6. 2019Fastened CROWN: Tightened Neural Network Robustness Certificatescited by 39
  7. 2018Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrationscited by 38
  8. 2020Parallelization Techniques for Verifying Neural Networkscited by 36
  9. 2021Towards Scalable Verification of Deep Reinforcement Learningcited by 33
  10. 2022Verifying Learning-Based Robotic Navigation Systemscited by 33
  11. 2021Direct Behavior Specification via Constrained Reinforcement Learningcited by 31
  12. 2019A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithmscited by 27
  13. 2020Global Optimization of Objective Functions Represented by ReLU Networkscited by 24
  14. 2021Safe Reinforcement Learning using Formal Verification for Tissue Retraction in Autonomous Robotic-Assisted Surgerycited by 24
  15. 2022An Abstraction-Refinement Approach to Verifying Convolutional Neural Networkscited by 24
  16. 2022Efficient Neural Network Analysis with Sum-of-Infeasibilitiescited by 24
  17. 2019Vision-based Navigation Using Deep Reinforcement Learningcited by 13
  18. 2021Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigationcited by 10
  19. 2022Scenario-Assisted Deep Reinforcement Learningcited by 8
  20. 2020Guarded Deep Learning using Scenario-Based Modelingcited by 6
  21. 2021Towards Repairing Scenario-Based Models with Rich Eventscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 7.

11 papers.

  1. 2023Formally Explaining Neural Networks within Reactive Systemscited by 24
  2. 2024Local vs. Global Interpretability: A Computational Complexity Perspectivecited by 23
  3. 2024Formally Verifying Deep Reinforcement Learning Controllers with Lyapunov Barrier Certificatescited by 16
  4. 2024Verification-Guided Shielding for Deep Reinforcement Learningcited by 13
  5. 2022veriFIRE: Verifying an Industrial, Learning-Based Wildfire Detection Systemcited by 12
  6. 2024Shield Synthesis for LTL Modulo Theoriescited by 12
  7. 2023Verifying Generalization in Deep Learningcited by 11
  8. 2024Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretationcited by 11
  9. 2024Analyzing Adversarial Inputs in Deep Reinforcement Learningcited by 7
  10. 2024Realizable Continuous-Space Shields for Safe Reinforcement Learningcited by 3
  11. 2023Enhancing Deep Learning with Scenario-Based Override Rules: a Case Studycited by 2