# Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients

arXiv: https://arxiv.org/abs/2406.15612 · HTML: https://mlsanity.com/paper/2406.15612/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [Efficient Risk-Averse Reinforcement Learning](https://mlsanity.com/paper/2205.05138/) (2022)
- [Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning](https://mlsanity.com/paper/2205.05614/) (2022)
- [Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning](https://mlsanity.com/paper/2206.14666/) (2022)
- [Robust Risk-Aware Option Hedging](https://mlsanity.com/paper/2303.15216/) (2023)
- [Bias-Corrected Peaks-Over-Threshold Estimation of the CVaR](https://mlsanity.com/paper/2103.05059/) (2021)
- [A policy gradient approach for optimization of smooth risk measures](https://mlsanity.com/paper/2202.11046/) (2022)
- [Automated Threshold Selection for Extreme Value Analysis via Goodness-of-Fit Tests with Application to Batched Return Level Mapping](https://mlsanity.com/paper/1604.02024/) (2016)

## Built on by (most cited first, 2 of 2)

- [EX-DRL: Hedging Against Heavy Losses with EXtreme Distributional Reinforcement Learning](https://mlsanity.com/paper/2408.12446/) (2024)
- [Training ML Models with Predictable Failures](https://mlsanity.com/paper/2605.15134/) (2026)

## Abstract (arXiv)

This paper tackles the problem of mitigating catastrophic risk (which is risk with very low frequency but very high severity) in the context of a sequential decision making process. This problem is particularly challenging due to the scarcity of observations in the far tail of the distribution of cumulative costs (negative rewards). A policy gradient algorithm is developed, that we call POTPG. It is based on approximations of the tail risk derived from extreme value theory. Numerical experiments highlight the out-performance of our method over common benchmarks, relying on the empirical distribution. An application to financial risk management, more precisely to the dynamic hedging of a financial option, is presented.