Data to 14 Oct 2026

Worst Cases Policy Gradients

arXiv 1911.03618 ↗First posted 9 Nov 2019cs.LG cs.AIcited by 52

Not on the map. Why?

Abstract (arXiv)

Recent advances in deep reinforcement learning have demonstrated the capability of learning complex control policies from many types of environments. When learning policies for safety-critical applications, it is essential to be sensitive to risks and avoid catastrophic events. Towards this goal, we propose an actor-critic framework that models the uncertainty of the future and simultaneously learns a policy based on that uncertainty model. Specifically, given a distribution of the future return for any state and action, we optimize policies for varying levels of conditional Value-at-Risk. The learned policy can map the same state to different actions depending on the propensity for risk. We demonstrate the effectiveness of our approach in the domain of driving simulations, where we learn maneuvers in two scenarios. Our learned controller can dynamically select actions along a continuous axis, where safe and conservative behaviors are found at one end while riskier behaviors are found at the other. Finally, when testing with very different simulation parameters, our risk-averse policies generalize significantly better compared to other reinforcement learning approaches.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 12; 2022: 8; 2023: 15; 2024: 5; 2025: 5; 2026: 6.

52 papers.

Sort:

By area:

2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
2021Conservative Offline Distributional Reinforcement Learningcited by 79
2021Risk-Averse Offline Reinforcement Learningcited by 65
2022Efficient Risk-Averse Reinforcement Learningcited by 43
2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 39
2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 29
2020Bayesian Robust Optimization for Imitation Learningcited by 28
2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
2023Improving Generalization of Alignment with Human Preferences through Group Invariant Learningcited by 25
2022Robust Policy Learning over Multiple Uncertainty Setscited by 18
2023Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learningcited by 14

38 more

2021Lyapunov Barrier Policy Optimizationcited by 12
2023An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradientcited by 12
2021Policy Gradient Bayesian Robust Optimization for Imitation Learningcited by 11
2021Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learningcited by 10
2023ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantagescited by 10
2023Adjustable Robust Reinforcement Learning for Online 3D Bin Packingcited by 10
2024RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashescited by 10
2024A Simple Mixture Policy Parameterization for Improving Sample Efficiency of CVaR Optimizationcited by 7
2025Risk-Averse Finetuning of Large Language Modelscited by 7
2022Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learningcited by 6
2022Risk-Sensitive Policy with Distributional Reinforcement Learningcited by 6
2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
2023On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economicscited by 6
2025Return Capping: Sample-Efficient CVaR Policy Gradient Optimisationcited by 6
2021Planning for Risk-Aversion and Expected Value in MDPscited by 5
2023Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theorycited by 5
2023TRC: Trust Region Conditional Value at Risk for Safe Reinforcement Learningcited by 5
2021DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Interventioncited by 4
2025Safety Representations for Safer Policy Learningcited by 4
2021Learning to Be Cautiouscited by 3
2022GIN: Graph-based Interaction-aware Constraint Policy Optimization for Autonomous Drivingcited by 3
2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learningcited by 3
2022Distributional Reinforcement Learning for Scheduling of Chemical Production Processescited by 2
2023Is Risk-Sensitive Reinforcement Learning Properly Resolved?cited by 2
2023Absolute Policy Optimizationcited by 2
2024UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systemscited by 2
2024Diffusion Policies for Risk-Averse Behavior Modeling in Offline Reinforcement Learningcited by 2
2023Risk-sensitive Actor-free Policy via Convex Optimizationcited by 1
2023Risk-Aware Continuous Control with Neural Contextual Banditscited by 1
2026Safe Langevin Soft Actor Criticcited by 1
2026RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidancecited by 1
2023Forward-PECVaR Algorithm: Exact Evaluation for CVaR SSPscited by 0
2023Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planningcited by 0
2025DIAL: Distribution-Informed Adaptive Learning of Multi-Task Constraints for Safety-Critical Systemscited by 0
2025Ensemble RL through Classifier Models: Enhancing Risk-Return Trade-offs in Trading Strategiescited by 0
2026Stochastic Actor-Critic: Mitigating Overestimation via Temporal Aleatoric Uncertaintycited by 0
2026Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinitycited by 0

50 most cited of 52 All 52 (JSON)