# Practical Risk Measures in Reinforcement Learning

arXiv: https://arxiv.org/abs/1908.08379 · HTML: https://mlsanity.com/paper/1908.08379/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2019.

## Built on by (most cited first, 5 of 5)

- [Reinforcement Learning with Dynamic Convex Risk Measures](https://mlsanity.com/paper/2112.13414/) (2021)
- [Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning](https://mlsanity.com/paper/2206.14666/) (2022)
- [Deep Reinforcement Learning for Equal Risk Pricing and Hedging under Dynamic Expectile Risk Measures](https://mlsanity.com/paper/2109.04001/) (2021)
- [Anomalous State Sequence Modeling to Enhance Safety in Reinforcement Learning](https://mlsanity.com/paper/2407.19860/) (2024)
- [Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discounting](https://mlsanity.com/paper/2602.04131/) (2026)

## Abstract (arXiv)

Practical application of Reinforcement Learning (RL) often involves risk considerations. We study a generalized approximation scheme for risk measures, based on Monte-Carlo simulations, where the risk measures need not necessarily be coherent. We demonstrate that, even in simple problems, measures such as the variance of the reward-to-go do not capture the risk in a satisfactory manner. In addition, we show how a risk measure can be derived from model's realizations. We propose a neural architecture for estimating the risk and suggest the risk critic architecture that can be use to optimize a policy under general risk measures. We conclude our work with experiments that demonstrate the efficacy of our approach.