mlsanityData to 7 Oct 2026

Reward Certification for Policy Smoothed Reinforcement Learning

arXiv 2312.06436 ↗First posted 11 Dec 2023cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) has achieved remarkable success in safety-critical areas, but it can be weakened by adversarial attacks. Recent studies have introduced "smoothed policies" in order to enhance its robustness. Yet, it is still challenging to establish a provable guarantee to certify the bound of its total reward. Prior methods relied primarily on computing bounds using Lipschitz continuity or calculating the probability of cumulative reward above specific thresholds. However, these techniques are only suited for continuous perturbations on the RL agent's observations and are restricted to perturbations bounded by the l2-norm. To address these limitations, this paper proposes a general black-box certification method capable of directly certifying the cumulative reward of the smoothed policy under various lp-norm bounded perturbations. Furthermore, we extend our methodology to certify perturbations on action spaces. Our approach leverages f-divergence to measure the distinction between the original distribution and the perturbed distribution, subsequently determining the certification bound by solving a convex optimisation problem. We provide a comprehensive theoretical analysis and run sufficient experiments in multiple environments. Our results show that our method not only improves the certified lower bound of mean cumulative reward but also demonstrates better efficiency than state-of-the-art techniques.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 43 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

35 of 43 references.

References · 35

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  7. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  8. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  9. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  10. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  11. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  12. 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 241
23 more
  1. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  2. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  3. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  4. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  5. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  6. 2018Reachability Analysis of Deep Neural Networks with Provable Guaranteescited by 93
  7. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
  8. 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
  9. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 53
  10. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  11. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  12. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 39
  13. 2020Certifying Confidence via Randomized Smoothingcited by 36
  14. 2023Randomized Adversarial Training via Taylor Expansioncited by 28
  15. 2019Optimal Attacks on Reinforcement Learning Policiescited by 21
  16. 2021Adversarial Driving: Attacking End-to-End Autonomous Drivingcited by 19
  17. 2022PRoA: A Probabilistic Robustness Assessment against Functional Perturbationscited by 19
  18. 2019Walking on the Edge: Fast, Low-Distortion Adversarial Examplescited by 18
  19. 2021Sparse Adversarial Video Attacks with Spatial Transformationscited by 11
  20. 2023Towards Verifying the Geometric Robustness of Large-scale Neural Networkscited by 11
  21. 2023Reachability Analysis of Neural Network Control Systemscited by 8
  22. 20223DVerifier: Efficient Robustness Verification for 3D Point Cloud Modelscited by 5
  23. 2023RePreM: Representation Pre-training with Masked Model for Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 2; 2026: 2.

6 papers.

  1. 2025Safety of Embodied Navigation: A Surveycited by 9
  2. 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learningcited by 6
  3. 2024Robust Deep Reinforcement Learning against Adversarial Behavior Manipulationcited by 2
  4. 2025CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified Radius Maximizationcited by 1
  5. 2026Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbationscited by 0
  6. 2026Interpretable but Fragile? Robustness of Concept Bottlenecks under Geometric-Semantic Perturbationscited by 0