# On the Global Convergence of Risk-Averse Natural Policy Gradient Methods with Expected Conditional Risk Measures

arXiv: https://arxiv.org/abs/2301.10932 · HTML: https://mlsanity.com/paper/2301.10932/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 26 Jan 2023.

## Builds on (most cited first, 7 of 7)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [On the Global Convergence Rates of Softmax Policy Gradient Methods](https://mlsanity.com/paper/2005.06392/) (2020)
- [Bridging the Gap Between Value and Policy Based Reinforcement Learning](https://mlsanity.com/paper/1702.08892/) (2017)
- [Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularization](https://mlsanity.com/paper/2007.06558/) (2020)
- [Global Optimality Guarantees For Policy Gradient Methods](https://mlsanity.com/paper/1906.01786/) (2019)
- [Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model](https://mlsanity.com/paper/2005.12900/) (2020)
- [Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measures](https://mlsanity.com/paper/2301.05981/) (2023)

## Built on by (most cited first, 7 of 7)

- [Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity](https://mlsanity.com/paper/2306.11626/) (2023)
- [Root Cause Attribution of Delivery Risks via Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/2408.05860/) (2024)
- [Absolute Policy Optimization](https://mlsanity.com/paper/2310.13230/) (2023)
- [Actor-Critic Algorithm for Dynamic Expectile and CVaR](https://mlsanity.com/paper/2605.07857/) (2026)
- [On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning](https://mlsanity.com/paper/2405.14749/) (2024)
- [Reinforcement learning with an expectile-based objective](https://mlsanity.com/paper/2602.09300/) (2026)
- [Long-Term Sequential Decision Making under Risk](https://mlsanity.com/paper/2607.19914/) (2026)

## Abstract (arXiv)

Risk-sensitive reinforcement learning (RL) has become a popular tool for controlling the risk of uncertain outcomes and ensuring reliable performance in highly stochastic sequential decision-making problems. While it has been shown that policy gradient methods can find globally optimal policies in the risk-neutral setting, it remains unclear if the risk-averse variants enjoy the same global convergence guarantees. In this paper, we consider a class of dynamic time-consistent risk measures, named Expected Conditional Risk Measures (ECRMs), and derive natural policy gradient (NPG) updates for ECRMs-based RL problems. We provide global optimality and iteration complexity of the proposed risk-averse NPG algorithm with softmax parameterization and entropy regularization under both exact and inexact policy evaluation. Furthermore, we test our risk-averse NPG algorithm on a stochastic Cliffwalk environment to demonstrate the efficacy of our method.