# Improving Robustness via Risk Averse Distributional Reinforcement Learning

arXiv: https://arxiv.org/abs/2005.00585 · HTML: https://mlsanity.com/paper/2005.00585/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 28 of 28)

- [Risk-Averse Offline Reinforcement Learning](https://mlsanity.com/paper/2102.05371/) (2021)
- [Uncertainty Estimation for Language Reward Models](https://mlsanity.com/paper/2203.07472/) (2022)
- [Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods](https://mlsanity.com/paper/2111.03941/) (2021)
- [Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity](https://mlsanity.com/paper/2306.11626/) (2023)
- [Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Bounds](https://mlsanity.com/paper/2210.14051/) (2022)
- [Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation](https://mlsanity.com/paper/2104.03111/) (2021)
- [Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observations](https://mlsanity.com/paper/2109.08776/) (2021)
- [Robust Reinforcement Learning with Distributional Risk-averse formulation](https://mlsanity.com/paper/2206.06841/) (2022)
- [Estimation and Inference in Distributional Reinforcement Learning](https://mlsanity.com/paper/2309.17262/) (2023)
- [Automatic Risk Adaptation in Distributional Reinforcement Learning](https://mlsanity.com/paper/2106.06317/) (2021)
- [Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shifts](https://mlsanity.com/paper/2402.09992/) (2024)
- [Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2402.02665/) (2024)
- [Deep Reinforcement Learning for Equal Risk Pricing and Hedging under Dynamic Expectile Risk Measures](https://mlsanity.com/paper/2109.04001/) (2021)
- [Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control](https://mlsanity.com/paper/2207.13730/) (2022)
- [DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty](https://mlsanity.com/paper/2506.12622/) (2025)
- [ACReL: Adversarial Conditional value-at-risk Reinforcement Learning](https://mlsanity.com/paper/2109.09470/) (2021)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity](https://mlsanity.com/paper/2208.03066/) (2022)
- [PACER: A Fully Push-forward-based Distributional Reinforcement Learning Algorithm](https://mlsanity.com/paper/2306.06637/) (2023)
- [Diffusion Policies for Risk-Averse Behavior Modeling in Offline Reinforcement Learning](https://mlsanity.com/paper/2403.17646/) (2024)
- [Beyond Bayes-optimality: meta-learning what you know you don't know](https://mlsanity.com/paper/2209.15618/) (2022)
- [Policy Evaluation in Distributional LQR](https://mlsanity.com/paper/2303.13657/) (2023)
- [On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning](https://mlsanity.com/paper/2405.14749/) (2024)
- [Applications of Reinforcement Learning in Deregulated Power Market: A Comprehensive Review](https://mlsanity.com/paper/2205.08369/) (2022)
- [Uniform Pessimistic Risk and its Optimal Portfolio](https://mlsanity.com/paper/2303.07158/) (2023)
- [DRL-ORA: Distributional Reinforcement Learning with Online Risk Adaption](https://mlsanity.com/paper/2310.05179/) (2023)
- [Risk-averse policies for natural gas futures trading using distributional reinforcement learning](https://mlsanity.com/paper/2501.04421/) (2025)
- [Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertainty](https://mlsanity.com/paper/2602.20729/) (2026)

## Abstract (arXiv)

One major obstacle that precludes the success of reinforcement learning in real-world applications is the lack of robustness, either to model uncertainties or external disturbances, of the trained policies. Robustness is critical when the policies are trained in simulations instead of real world environment. In this work, we propose a risk-aware algorithm to learn robust policies in order to bridge the gap between simulation training and real-world implementation. Our algorithm is based on recently discovered distributional RL framework. We incorporate CVaR risk measure in sample based distributional policy gradients (SDPG) for learning risk-averse policies to achieve robustness against a range of system disturbances. We validate the robustness of risk-aware SDPG on multiple environments.