mlsanityData to 14 Oct 2026

Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation

arXiv 2301.11374 ↗First posted 26 Jan 2023cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

We present a reinforcement learning (RL) framework in which the learned policy comes with a machine-checkable certificate of provable adversarial robustness. Our approach, called CAROL, learns a model of the environment. In each learning iteration, it uses the current version of this model and an external abstract interpreter to construct a differentiable signal for provable robustness. This signal is used to guide learning, and the abstract interpretation used to construct it directly leads to the robustness certificate returned at convergence. We give a theoretical analysis that bounds the worst-case accumulative reward of CAROL. We also experimentally evaluate CAROL on four MuJoCo environments with continuous state and action spaces. On these tasks, CAROL learns policies that, when contrasted with policies from the state-of-the-art robust RL algorithms, exhibit: (i) markedly enhanced certified performance lower bounds; and (ii) comparable performance under empirical adversarial attacks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

33 of 47 references.

References · 33

  1. 2016OpenAI Gymcited by 1,736
  2. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  3. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  4. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  5. 2017Robust Adversarial Reinforcement Learningcited by 451
  6. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
  7. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
  8. 2017Adversarial Attacks on Neural Network Policiescited by 320
  9. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  10. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  11. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  12. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 225
21 more
  1. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  2. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  3. 2022Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methodscited by 151
  4. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  5. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
  6. 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 94
  7. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 78
  8. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 75
  9. 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 55
  10. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  11. 2021MBRL-Lib: A Modular Library for Model-based Reinforcement Learningcited by 42
  12. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 42
  13. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 40
  14. 2019Control Regularization for Reduced Variance Reinforcement Learningcited by 37
  15. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 35
  16. 2019Online Robustness Training for Deep Reinforcement Learningcited by 26
  17. 2020Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learningcited by 24
  18. 2019Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functionscited by 21
  19. 2019Optimal Attacks on Reinforcement Learning Policiescited by 21
  20. 2020Automatic Perturbation Analysis for Scalable Certified Robustness and Beyondcited by 10
  21. 2022Safe Neurosymbolic Learning with Differentiable Symbolic Executioncited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024Canopy: Property-Driven Learning for Congestion Controlcited by 2
  2. 2024Training Verifiably Robust Agents Using Set-Based Reinforcement Learningcited by 0
  3. 2026FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policiescited by 0