mlsanityData to 7 Oct 2026

Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning

arXiv 2004.10888 ↗First posted 22 Apr 2020cs.LG cs.AI stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

We present a mean-variance policy iteration (MVPI) framework for risk-averse control in a discounted infinite horizon MDP optimizing the variance of a per-step reward random variable. MVPI enjoys great flexibility in that any policy evaluation method and risk-neutral control method can be dropped in for risk-averse control off the shelf, in both on- and off-policy settings. This flexibility reduces the gap between risk-neutral control and risk-averse control and is achieved by working on a novel augmented MDP directly. We propose risk-averse TD3 as an example instantiating MVPI, which outperforms vanilla TD3 and many previous risk-averse control methods in challenging Mujoco robot simulation tasks under a risk-aware performance metric. This risk-averse TD3 is the first to introduce deterministic policies and off-policy learning into risk-averse reinforcement learning, both of which are key to the performance boost we show in Mujoco domains.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 7; 2023: 3; 2024: 5; 2025: 6; 2026: 4.

28 papers.

Sort:

By area:

  1. 2021Average-Reward Off-Policy Policy Evaluation with Function Approximationcited by 30
  2. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
  3. 2022A Game-Theoretic Framework for Managing Risk in Multi-Agent Systemscited by 13
  4. 2021Risk-Aware Transfer in Reinforcement Learning using Successor Featurescited by 12
  5. 2023An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradientcited by 12
  6. 2021Policy Gradient Bayesian Robust Optimization for Imitation Learningcited by 11
  7. 2022Risk-Sensitive Policy with Distributional Reinforcement Learningcited by 6
  8. 2024Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shiftscited by 6
  9. 2022A unified algorithm framework for mean-variance optimization in discounted Markov decision processescited by 3
  10. 2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
  11. 2023Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterioncited by 3
  12. 2024Tractable Equilibrium Computation in Markov Games through Risk Aversioncited by 3
16 more
  1. 2025Average Reward Reinforcement Learning for Wireless Radio Resource Managementcited by 3
  2. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functionscited by 2
  3. 2025Risk-sensitive Actor-Critic with Static Spectral Risk Measures for Online and Offline Reinforcement Learningcited by 2
  4. 2026LAD: Learning Advantage Distribution for Reasoningcited by 2
  5. 2026SAT: Sequential Agent Tuning for Coordinator Free Plug and Play Multi-LLM Training with Monotonic Improvement Guaranteescited by 2
  6. 2022Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learningcited by 1
  7. 2023Global Algorithms for Mean-Variance Optimization in Markov Decision Processescited by 1
  8. 2024Bootstrapping Expectiles in Reinforcement Learningcited by 1
  9. 2024A Finite-Sample Analysis of an Actor-Critic Algorithm for Mean-Variance Optimization in a Discounted MDPcited by 1
  10. 2024TRAM: Test-Time Risk Adaptation with Mixture of Agentscited by 1
  11. 2022STOPS: Short-Term-based Volatility-controlled Policy Search and its Global Convergencecited by 0
  12. 2025Measures of Variability for Risk-averse Policy Gradientcited by 0
  13. 2025Risk-Averse Total-Reward Reinforcement Learningcited by 0
  14. 2025Sharpe Ratio Optimization in Markov Decision Processescited by 0
  15. 2026Risk-Sensitive Exponential Actor Criticcited by 0
  16. 2026A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learningcited by 0