mlsanityData to 14 Oct 2026

Towards Scalable Verification of Deep Reinforcement Learning

arXiv 2105.11931 ↗First posted 25 May 2021cs.LG cs.NI math.OCcited by 33

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have gained significant popularity in recent years, becoming the state of the art in a variety of domains. In particular, deep reinforcement learning (DRL) has recently been employed to train DNNs that realize control policies for various types of real-world systems. In this work, we present the whiRL 2.0 tool, which implements a new approach for verifying complex properties of interest for DRL systems. To demonstrate the benefits of whiRL 2.0, we apply it to case studies from the communication networks domain that have recently been used to motivate formal verification of DRL systems, and which exhibit characteristics that are conducive for scalable verification. We propose techniques for performing k-induction and semi-automated invariant inference on such systems, and leverage these techniques for proving safety and liveness properties that were previously impossible to verify due to the scalability barriers of prior approaches. Furthermore, we show how our proposed techniques provide insights into the inner workings and the generalizability of DRL systems. whiRL 2.0 is publicly available online.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 20 of its 64 references that are in the map. First posted on arXiv 25 May 2021.

Builds on

20 of 64 references.

References · 20

  1. 2013Intriguing properties of neural networkscited by 5,155
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2016End to End Learning for Self-Driving Carscited by 1,009
  4. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
  5. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
  6. 2017Deep Reinforcement Learning: An Overviewcited by 333
  7. 2016Safety Verification of Deep Neural Networkscited by 263
  8. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 255
  9. 2018Formal Security Analysis of Neural Networks using Symbolic Intervalscited by 185
  10. 2019An Abstraction-Based Framework for Neural Network Verificationcited by 68
  11. 2020An SMT-Based Approach for Verifying Binarized Neural Networkscited by 47
  12. 2018Formal Verification of Neural Network Controlled Autonomous Systemscited by 41
8 more
  1. 2020Parallelization Techniques for Verifying Neural Networkscited by 36
  2. 2017Towards Proving the Adversarial Robustness of Deep Neural Networkscited by 26
  3. 2020Verifying Recurrent Neural Networks using Invariant Inferencecited by 26
  4. 2020Global Optimization of Objective Functions Represented by ReLU Networkscited by 24
  5. 2020Static Neural Compiler Optimization via Deep Reinforcement Learningcited by 14
  6. 2018Toward Scalable Verification for Safety-Critical Deep Networkscited by 12
  7. 2020RAID: Randomized Adversarial-Input Detection for Neural Networkscited by 8
  8. 2017Deep Reinforcement Learning for Multi-Resource Multi-Machine Job Schedulingcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 11; 2023: 4; 2024: 13; 2025: 2; 2026: 1.

33 papers.

Sort:

By area:

  1. 2024Marabou 2.0: A Versatile Formal Analyzer of Neural Networkscited by 81
  2. 2022Towards Formal XAI: Formally Approximate Minimal Explanations of Neural Networkscited by 42
  3. 2022Verifying Learning-Based Robotic Navigation Systemscited by 33
  4. 2022An Abstraction-Refinement Approach to Verifying Convolutional Neural Networkscited by 24
  5. 2023Formally Explaining Neural Networks within Reactive Systemscited by 24
  6. 2024Local vs. Global Interpretability: A Computational Complexity Perspectivecited by 23
  7. 2022Neural Network Verification with Proof Productioncited by 22
  8. 2021Pruning and Slicing Neural Networks using Formal Verificationcited by 20
  9. 2022Scalable Verification of GNN-based Job Schedulerscited by 19
  10. 2022Verification-Aided Deep Ensemble Selectioncited by 18
  11. 2024Formally Verifying Deep Reinforcement Learning Controllers with Lyapunov Barrier Certificatescited by 16
  12. 2021Minimal Multi-Layer Modifications of Deep Neural Networkscited by 14
21 more
  1. 2024Verification-Guided Shielding for Deep Reinforcement Learningcited by 13
  2. 2022veriFIRE: Verifying an Industrial, Learning-Based Wildfire Detection Systemcited by 12
  3. 2024Shield Synthesis for LTL Modulo Theoriescited by 12
  4. 2022Constrained Reinforcement Learning for Robotics via Scenario-Based Programmingcited by 11
  5. 2023Verifying Generalization in Deep Learningcited by 11
  6. 2024Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretationcited by 11
  7. 2022Tighter Abstract Queries in Neural Network Verificationcited by 8
  8. 2024Analyzing Adversarial Inputs in Deep Reinforcement Learningcited by 7
  9. 2022On Optimizing Back-Substitution Methods for Neural Network Verificationcited by 6
  10. 2024Unifying Qualitative and Quantitative Safety Verification of DNN-Controlled Systemscited by 6
  11. 2024Towards Formal Verification of Deep Neural Networks for Object Detectioncited by 4
  12. 2024Safe and Reliable Training of Learning-Based Aerospace Controllerscited by 4
  13. 2022Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous Systemscited by 3
  14. 2023DelBugV: Delta-Debugging Neural Network Verifierscited by 3
  15. 2025PICID: Proof-Driven Clause Learning in Neural Network Verificationcited by 3
  16. 2024Composing Reinforcement Learning Policies, with Formal Guaranteescited by 2
  17. 2024Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learningcited by 1
  18. 2023Compositional Inductive Invariant Based Verification of Neural Network Controlled Systemscited by 0
  19. 2024Reinfier and Reintrainer: Verification and Interpretation-Driven Safe Deep Reinforcement Learning Frameworkscited by 0
  20. 2025Efficient Dynamic Shielding for Parametric Safety Specificationscited by 0
  21. 2026Talking with Verifiers: Automatic Specification Generation for Neural Network Verificationcited by 0