CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified Radius Maximization
Not on the map. Why?
Abstract (arXiv)
Deep reinforcement learning (DRL) has gained widespread adoption in control and decision-making tasks due to its strong performance in dynamic environments. However, DRL agents are vulnerable to noisy observations and adversarial attacks, and concerns about the adversarial robustness of DRL systems have emerged. Recent efforts have focused on addressing these robustness issues by establishing rigorous theoretical guarantees for the returns achieved by DRL agents in adversarial settings. Among these approaches, policy smoothing has proven to be an effective and scalable method for certifying the robustness of DRL agents. Nevertheless, existing certifiably robust DRL relies on policies trained with simple Gaussian augmentations, resulting in a suboptimal trade-off between certified robustness and certified return. To address this issue, we introduce a novel paradigm dubbed textttCertified-rtextttAdius-textttMaximizing textttPolicy (textttCAMP) training. textttCAMP is designed to enhance DRL policies, achieving better utility without compromising provable robustness. By leveraging the insight that the global certified radius can be derived from local certified radii based on training-time statistics, textttCAMP formulates a surrogate loss related to the local certified radius and optimizes the policy guided by this surrogate loss. We also introduce textitpolicy imitation as a novel technique to stabilize textttCAMP training. Experimental results demonstrate that textttCAMP significantly improves the robustness-return trade-off across various tasks. Based on the results, textttCAMP can achieve up to twice the certified expected return compared to that of baselines. Our code is available at https://github.com/NeuralSec/camp-robust-rl.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 51 of its 71 references that are in the map. First posted on arXiv 29 Jan 2025.
Builds on
51 of 71 references.
References · 51
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2015End-to-End Training of Deep Visuomotor Policiescited by 847
- 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
- 2017Robust Adversarial Reinforcement Learningcited by 439
- 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
- 2017Adversarial Attacks on Neural Network Policiescited by 314
- 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
- 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
- 2018Certified Adversarial Robustness with Additive Noisecited by 178
38 more
- 2017Parameter Space Noise for Explorationcited by 158
- 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
- 2020Randomized Smoothing of All Shapes and Sizescited by 146
- 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
- 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
- 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
- 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
- 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 83
- 2020Consistency Regularization for Certified Robustness of Smoothed Classifierscited by 73
- 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
- 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 62
- 2017Adversarial examples for generative modelscited by 58
- 2020Curse of Dimensionality on Randomized Smoothing for Certifiable Robustnesscited by 56
- 2017End-to-End Task-Completion Neural Dialogue Systemscited by 52
- 2017Reinforcement Learning under Model Mismatchcited by 52
- 2020Certified Defense to Image Transformations via Randomized Smoothingcited by 51
- 2020TSS: Transformation-Specific Smoothing for Robustness Certificationcited by 49
- 2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49
- 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
- 2020Robust Reinforcement Learning via Adversarial training with Langevin Dynamicscited by 45
- 2020Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Frameworkcited by 45
- 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
- 2020On the Robustness of Cooperative Multi-Agent Reinforcement Learningcited by 39
- 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 39
- 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
- 2020Certifying Confidence via Randomized Smoothingcited by 36
- 2021Ensemble Bootstrapping for Q-Learningcited by 30
- 2021Intriguing Properties of Input-dependent Randomized Smoothingcited by 24
- 2022GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothingcited by 23
- 2022Double Sampling Randomized Smoothingcited by 22
- 2023Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustnesscited by 18
- 2024Provably Unlearnable Data Examplescited by 18
- 2022Double Bubble, Toil and Trouble: Enhancing Certified Robustness through Transitivitycited by 15
- 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
- 2023Reward Certification for Policy Smoothed Reinforcement Learningcited by 6
- 2022Policy Learning for Robust Markov Decision Process with a Mismatched Generative Modelcited by 4
- 2024Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agentscited by 4
- 2024Transferring human emotions to robot motions using Neural Policy Style Transfercited by 1
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.