mlsanityData to 7 Oct 2026

Robust Risk-Aware Reinforcement Learning

arXiv 2108.10403 ↗First posted 23 Aug 2021cs.LG q-fin.CP q-fin.PMcited by 17

Not on the map. Why?

Abstract (arXiv)

We present a reinforcement learning (RL) approach for robust optimisation of risk-aware performance criteria. To allow agents to express a wide variety of risk-reward profiles, we assess the value of a policy using rank dependent expected utility (RDEU). RDEU allows the agent to seek gains, while simultaneously protecting themselves against downside risk. To robustify optimal policies against model uncertainty, we assess a policy not by its distribution, but rather, by the worst possible distribution that lies within a Wasserstein ball around it. Thus, our problem formulation may be viewed as an actor/agent choosing a policy (the outer problem), and the adversary then acting to worsen the performance of that strategy (the inner problem). We develop explicit policy gradient formulae for the inner and outer problems, and show its efficacy on three prototypical financial problems: robust portfolio allocation, optimising a benchmark, and statistical arbitrage.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 9 of its 22 references that are in the map. First posted on arXiv 23 Aug 2021.

Builds on

9 of 22 references.

References · 9

  1. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  2. 2013Risk-sensitive Reinforcement Learningcited by 129
  3. 2015Policy Gradient for Coherent Risk Measurescited by 74
  4. 2020Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regretcited by 57
  5. 2019Wasserstein Robust Reinforcement Learningcited by 50
  6. 2020Entropy Regularization for Mean Field Games with Learningcited by 50
  7. 2019Distributionally Robust Reinforcement Learningcited by 40
  8. 2021On the Convergence and Optimality of Policy Gradient for Markov Coherent Riskcited by 19
  9. 2020Exploratory LQG Mean Field Games with Entropy Regularizationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 4; 2024: 2; 2025: 4; 2026: 1.

17 papers.

Sort:

By area:

  1. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  2. 2021Learning Stochastic Shortest Path with Linear Function Approximationcited by 23
  3. 2023Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexitycited by 16
  4. 2022Robust Reinforcement Learning with Distributional Risk-averse formulationcited by 9
  5. 2022Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learningcited by 9
  6. 2023Robust Risk-Aware Option Hedgingcited by 9
  7. 2022A Risk-Sensitive Approach to Policy Optimizationcited by 6
  8. 2025TW-CRL: Time-Weighted Contrastive Reward Learning for Efficient Inverse Reinforcement Learningcited by 2
  9. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functionscited by 2
  10. 2025Risk-aware Direct Preference Optimization under Nested Risk Measurecited by 2
  11. 2022Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimizationcited by 1
  12. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 1
5 more
  1. 2023Risk-Averse MDPs under Reward Ambiguitycited by 0
  2. 2023Towards Theoretical Understanding of Data-Driven Policy Refinementcited by 0
  3. 2024Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learningcited by 0
  4. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Setscited by 0
  5. 2026Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deploymentcited by 0