mlsanityData to 7 Oct 2026

A Bayesian Approach to Robust Reinforcement Learning

arXiv 1905.08188 ↗First posted 20 May 2019cs.LG cs.AI stat.MLcited by 32

Not on the map. Why?

Abstract (arXiv)

Robust Markov Decision Processes (RMDPs) intend to ensure robustness with respect to changing or adversarial system behavior. In this framework, transitions are modeled as arbitrary elements of a known and properly structured uncertainty set and a robust optimal policy can be derived under the worst-case scenario. In this study, we address the issue of learning in RMDPs using a Bayesian approach. We introduce the Uncertainty Robust Bellman Equation (URBE) which encourages safe exploration for adapting the uncertainty set to new observations while preserving robustness. We propose a URBE-based algorithm, DQN-URBE, that scales this method to higher dimensional domains. Our experiments show that the derived URBE-based strategy leads to a better trade-off between less conservative solutions and robustness in the presence of model misspecification. In addition, we show that the DQN-URBE algorithm can adapt significantly faster to changing dynamics online compared to existing robust techniques with fixed uncertainty sets.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 8; 2024: 6; 2025: 4; 2026: 2.

32 papers.

Sort:

By area:

  1. 2022Robust Reinforcement Learning using Offline Datacited by 95
  2. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 82
  3. 2022Constrained Policy Optimization via Bayesian World Modelscited by 55
  4. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  5. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
  6. 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 28
  7. 2021Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptoticscited by 27
  8. 2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
  9. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  10. 2021When should agents explore?cited by 20
  11. 2023Distributionally Robust Model-based Reinforcement Learning with Large State Spacescited by 17
  12. 2022Robust Anytime Learning of Markov Decision Processescited by 16
20 more
  1. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  2. 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptionscited by 8
  3. 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
  4. 2024Certifiably Robust Policies for Uncertain Parametric Environmentscited by 6
  5. 2023Minimax-Bayes Reinforcement Learningcited by 5
  6. 2023Value-Distributional Model-Based Reinforcement Learningcited by 5
  7. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidencecited by 5
  8. 2022Policy Learning for Robust Markov Decision Process with a Mismatched Generative Modelcited by 4
  9. 2022A Simulation Environment and Reinforcement Learning Method for Waste Reductioncited by 3
  10. 2023Model-based Offline Policy Optimization with Adversarial Networkcited by 3
  11. 2021Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific researchcited by 1
  12. 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learningcited by 1
  13. 2025Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatismcited by 1
  14. 2026RE-SAC: Disentangling aleatoric and epistemic risks in bus fleet control: A stable and robust ensemble DRL approachcited by 1
  15. 2023An Autonomous Non-monolithic Agent with Multi-mode Exploration based on Options Frameworkcited by 0
  16. 2023On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithmcited by 0
  17. 2024Percentile Criterion Optimization in Offline Reinforcement Learningcited by 0
  18. 2024Survival of the Fittest: Evolutionary Adaptation of Policies for Environmental Shiftscited by 0
  19. 2025Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systemscited by 0
  20. 2026Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Beliefcited by 0