mlsanityData to 14 Oct 2026

Robust Reinforcement Learning with Distributional Risk-averse formulation

arXiv 2206.06841 ↗First posted 14 Jun 2022cs.LG math.OC stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Robust Reinforcement Learning tries to make predictions more robust to changes in the dynamics or rewards of the system. This problem is particularly important when the dynamics and rewards of the environment are estimated from the data. In this paper, we approximate the Robust Reinforcement Learning constrained with a Phi-divergence using an approximate Risk-Averse formulation. We show that the classical Reinforcement Learning formulation can be robustified using standard deviation penalization of the objective. Two algorithms based on Distributional Reinforcement Learning, one for discrete and one for continuous action spaces are proposed and tested in a classical Gym environment to demonstrate the robustness of the algorithms.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

33 of 44 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  6. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  7. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  8. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 335
  9. 2019A Theory of Regularized Markov Decision Processescited by 286
  10. 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 183
  11. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  12. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
21 more
  1. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
  2. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  3. 2021Conservative Offline Distributional Reinforcement Learningcited by 79
  4. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  5. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  6. 2019Wasserstein Robust Reinforcement Learningcited by 54
  7. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 47
  8. 2019Distributionally Robust Reinforcement Learningcited by 42
  9. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 41
  10. 2019Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPscited by 40
  11. 2020Distributional Robustness and Regularization in Reinforcement Learningcited by 40
  12. 2021Regularized Policies are Reward Robustcited by 32
  13. 2020Improving Robustness via Risk Averse Distributional Reinforcement Learningcited by 28
  14. 2020Scalable First-Order Methods for Robust MDPscited by 25
  15. 2022Your Policy Regularizer is Secretly an Adversarycited by 21
  16. 2022Efficient Policy Iteration for Robust Markov Decision Processes via Regularizationcited by 20
  17. 2021Robust Risk-Aware Reinforcement Learningcited by 18
  18. 2020First-Order Methods for Wasserstein Distributionally Robust MDPcited by 17
  19. 2021GMAC: A Distributional Perspective on Actor-Critic Frameworkcited by 16
  20. 2021Variance Penalized On-Policy and Off-Policy Actor-Criticcited by 10
  21. 2021Safe Distributional Reinforcement Learningcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 0; 2026: 3.

10 papers.

  1. 2026Maximum Likelihood Reinforcement Learningcited by 36
  2. 2023Towards Minimax Optimality of Model-based Robust Reinforcement Learningcited by 22
  3. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 3
  4. 2026Tail-Likelihood Reinforcement Learningcited by 3
  5. 2024A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learningcited by 2
  6. 2024A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertaintycited by 2
  7. 2024Time-Constrained Robust MDPscited by 2
  8. 2024RRLS : Robust Reinforcement Learning Suitecited by 2
  9. 2024Bootstrapping Expectiles in Reinforcement Learningcited by 1
  10. 2026Formal Synthesis of Certifiably Robust Neural Lyapunov-Barrier Certificatescited by 0