A policy gradient approach for optimization of smooth risk measures
Not on the map. Why?
Abstract (arXiv)
We propose policy gradient algorithms for solving a risk-sensitive reinforcement learning (RL) problem in on-policy as well as off-policy settings. We consider episodic Markov decision processes, and model the risk using the broad class of smooth risk measures of the cumulative discounted reward. We propose two template policy gradient algorithms that optimize a smooth risk measure in on-policy and off-policy RL settings, respectively. We derive non-asymptotic bounds that quantify the rate of convergence of our proposed algorithms to a stationary point of the smooth risk measure. As special cases, we establish that our algorithms apply to optimization of mean-variance and distortion risk measures, respectively.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 11 of its 37 references that are in the map. First posted on arXiv 22 Feb 2022.
Builds on
11 of 37 references.
References · 11
- 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
- 2019Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policiescited by 117
- 2015An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedbackcited by 105
- 2018Stochastic Variance-Reduced Policy Gradientcited by 103
- 2015Policy Gradient for Coherent Risk Measurescited by 74
- 2015Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and Controlcited by 34
- 2014Variance-Constrained Actor-Critic Algorithms for Discounted and Average Reward MDPscited by 28
- 2014Policy Gradients for CVaR-Constrained MDPscited by 20
- 2021Spectral risk-based learning using unbounded lossescited by 8
- 2021Policy Gradient Methods for Distortion Risk Measurescited by 6
- 2021Smoothed functional-based gradient algorithms for off-policy reinforcement learning: A non-asymptotic viewpointcited by 5
Built on by
Citing papers per year: 2024: 2.
2 papers.