mlsanityData to 7 Oct 2026

Robust Offline Reinforcement Learning -- Certify the Confidence Interval

arXiv 2309.16631 ↗First posted 28 Sep 2023cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Currently, reinforcement learning (RL), especially deep RL, has received more and more attention in the research area. However, the security of RL has been an obvious problem due to the attack manners becoming mature. In order to defend against such adversarial attacks, several practical approaches are developed, such as adversarial training, data filtering, etc. However, these methods are mostly based on empirical algorithms and experiments, without rigorous theoretical analysis of the robustness of the algorithms. In this paper, we develop an algorithm to certify the robustness of a given policy offline with random smoothing, which could be proven and conducted as efficiently as ones without random smoothing. Experiments on different environments confirm the correctness of our algorithm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 36 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

30 of 36 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation23 Nov
  2. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothing6 Jun
  3. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks16 Mar
  4. 2022Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism11 Mar
  5. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learning5 Feb
  6. 2021Corruption-Robust Offline Reinforcement Learning11 Jun
  7. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  8. 2019Benchmarking Batch Deep Reinforcement Learning Algorithms3 Oct

Other references · 22

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  4. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  5. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  6. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  7. 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 187
  8. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  9. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  10. 2020GenDICE: Generalized Offline Estimation of Stationary Valuescited by 122
  11. 2020Reinforcement Learning via Fenchel-Rockafellar Dualitycited by 100
  12. 2020SoK: Certified Robustness for Deep Neural Networkscited by 100
10 more
  1. 2020GradientDICE: Rethinking Generalized Offline Estimation of Stationary Valuescited by 70
  2. 2020CoinDICE: Off-Policy Confidence Interval Estimationcited by 64
  3. 2020Robust Reinforcement Learning via Adversarial training with Langevin Dynamicscited by 45
  4. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 42
  5. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  6. 2021First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approachcited by 38
  7. 2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31
  8. 2022Robust Deep Reinforcement Learning through Bootstrapped Opportunistic Curriculumcited by 15
  9. 2021Corruption Robust Active Learningcited by 3
  10. 2020Randomized Policy Learning for Continuous State and Action MDPscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses
  2. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
  3. 2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation
  4. 2024Corruption Robust Offline Reinforcement Learning with Human Feedback
  5. 2023Corruption-Robust Offline Reinforcement Learning with General Function Approximation
  6. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity
  7. 2024Robust Offline Reinforcement Learning for Non-Markovian Decision Processes
  8. 2023Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness