mlsanityData to 14 Oct 2026

Reinforcement Learning with Dynamic Convex Risk Measures

arXiv 2112.13414 ↗First posted 26 Dec 2021cs.LG q-fin.CP q-fin.MFcited by 21

Not on the map. Why?

Abstract (arXiv)

We develop an approach for solving time-consistent risk-sensitive stochastic optimization problems using model-free reinforcement learning (RL). Specifically, we assume agents assess the risk of a sequence of random variables using dynamic convex risk measures. We employ a time-consistent dynamic programming principle to determine the value of a particular policy, and develop policy gradient update rules that aid in obtaining optimal policies. We further develop an actor-critic style algorithm using neural networks to optimize over policies. Finally, we demonstrate the performance and flexibility of our approach by applying it to three optimization problems: statistical arbitrage trading strategies, financial hedging, and obstacle avoidance robot control.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 24 of its 70 references that are in the map. First posted on arXiv 26 Dec 2021.

Builds on

24 of 70 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  3. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  4. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  5. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
  6. 2013Risk-sensitive Reinforcement Learningcited by 130
  7. 2015Policy Gradient for Coherent Risk Measurescited by 76
  8. 2020Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizoncited by 50
  9. 2020A generative adversarial network approach to calibration of local stochastic volatility modelscited by 29
  10. 2014Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Banditscited by 26
  11. 2021On the Convergence and Optimality of Policy Gradient for Markov Coherent Riskcited by 20
  12. 2021Deep Learning for Mean Field Games and Mean Field Control with Applications to Financecited by 20
12 more
  1. 2019Entropic Risk Measure in Policy Searchcited by 19
  2. 2020Markov Decision Processes with Recursive Risk Measurescited by 18
  3. 2020Constrained Risk-Averse Markov Decision Processescited by 15
  4. 2019Deep Q-Learning for Nash Equilibria: Nash-DQNcited by 14
  5. 2020Distributionally Robust Markov Decision Processes and their Connection to Risk Measurescited by 14
  6. 2019Deep Fictitious Play for Stochastic Differential Gamescited by 13
  7. 2020Risk-Averse Learning by Temporal Difference Methodscited by 13
  8. 2020Minimizing Spectral Risk Measures Applied to Markov Decision Processescited by 12
  9. 2017Approximate Value Iteration for Risk-aware Markov Decision Processescited by 9
  10. 2021Arbitrage-Free Implied Volatility Surface Generation with Variational Autoencoderscited by 9
  11. 2019Practical Risk Measures in Reinforcement Learningcited by 5
  12. 2021Deep Learning for Principal-Agent Mean Field Gamescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 1; 2025: 5; 2026: 7.

21 papers.

Sort:

By area:

  1. 2023Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexitycited by 17
  2. 2022Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learningcited by 9
  3. 2023Robust Risk-Aware Option Hedgingcited by 9
  4. 2023Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measurescited by 8
  5. 2023Risk-sensitive Markov Decision Process and Learning under General Utility Functionscited by 7
  6. 2023FuNVol: A Multi-Asset Implied Volatility Market Simulator using Functional Principal Components and Neural SDEscited by 6
  7. 2025Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learningcited by 6
  8. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 3
  9. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functionscited by 2
  10. 2025Risk-sensitive Actor-Critic with Static Spectral Risk Measures for Online and Offline Reinforcement Learningcited by 2
  11. 2022Risk-Aware Linear Bandits: Theory and Applications in Smart Order Routingcited by 1
  12. 2022Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimizationcited by 1
9 more
  1. 2026Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discountingcited by 1
  2. 2025Periodic evaluation of defined-contribution pension fund: A dynamic risk measure approachcited by 0
  3. 2025A Data-Driven Interpolation Method on Smooth Manifolds via Diffusion Processes and Voronoi Tessellationscited by 0
  4. 2026Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinitycited by 0
  5. 2026Markov Decision Processes of the Third Kind: Learning Distributions by Policy Gradient Descentcited by 0
  6. 2026Reinforcement learning with an expectile-based objectivecited by 0
  7. 2026A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learningcited by 0
  8. 2026Long-Term Sequential Decision Making under Riskcited by 0
  9. 2026Online Policy Evaluation for MDPs with Dynamic UBSR Measurescited by 0