mlsanityData to 14 Oct 2026

Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning

arXiv 2206.14666 ↗First posted 29 Jun 2022cs.LG q-fin.CP q-fin.PMcited by 9

Not on the map. Why?

Abstract (arXiv)

We propose a novel framework to solve risk-sensitive reinforcement learning (RL) problems where the agent optimises time-consistent dynamic spectral risk measures. Based on the notion of conditional elicitability, our methodology constructs (strictly consistent) scoring functions that are used as penalizers in the estimation procedure. Our contribution is threefold: we (i) devise an efficient approach to estimate a class of dynamic spectral risk measures with deep neural networks, (ii) prove that these dynamic spectral risk measures may be approximated to any arbitrary accuracy using deep neural networks, and (iii) develop a risk-sensitive actor-critic algorithm that uses full episodes and does not require any additional nested transitions. We compare our conceptually improved reinforcement learning algorithm with the nested simulation approach and illustrate its performance in two settings: statistical arbitrage and portfolio allocation on both simulated and real data.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 20 of its 77 references that are in the map. First posted on arXiv 29 Jun 2022.

Builds on

20 of 77 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  4. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  5. 2013Risk-sensitive Reinforcement Learningcited by 130
  6. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  7. 2018Convergence of the Deep BSDE Method for Coupled FBSDEscited by 43
  8. 2018Beyond expectation: Deep joint mean and quantile regression for spatio-temporal problemscited by 32
  9. 2021Reinforcement Learning with Dynamic Convex Risk Measurescited by 21
  10. 2021On the Convergence and Optimality of Policy Gradient for Markov Coherent Riskcited by 20
  11. 2019Entropic Risk Measure in Policy Searchcited by 19
  12. 2020Markov Decision Processes with Recursive Risk Measurescited by 18
8 more
  1. 2021Robust Risk-Aware Reinforcement Learningcited by 18
  2. 2020Constrained Risk-Averse Markov Decision Processescited by 15
  3. 2020Risk-Averse Learning by Temporal Difference Methodscited by 13
  4. 2020Minimizing Spectral Risk Measures Applied to Markov Decision Processescited by 12
  5. 2022Sensitivity Measures Based on Scoring Functionscited by 6
  6. 2019Practical Risk Measures in Reinforcement Learningcited by 5
  7. 2021Deep Reinforcement Learning for Equal Risk Pricing and Hedging under Dynamic Expectile Risk Measurescited by 3
  8. 2021Deep Quantile and Deep Composite Model Regressioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 0; 2026: 2.

9 papers.

  1. 2023Robust Risk-Aware Option Hedgingcited by 9
  2. 2023Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measurescited by 8
  3. 2023FuNVol: A Multi-Asset Implied Volatility Market Simulator using Functional Principal Components and Neural SDEscited by 6
  4. 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measurescited by 3
  5. 2023Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioningcited by 2
  6. 2024Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradientscited by 2
  7. 2026Actor-Critic Algorithm for Dynamic Expectile and CVaRcited by 2
  8. 2024Kullback-Leibler Barycentre of Stochastic Processescited by 0
  9. 2026Online Policy Evaluation for MDPs with Dynamic UBSR Measurescited by 0