mlsanityData to 7 Oct 2026

Safe Distributional Reinforcement Learning

arXiv 2102.13446 ↗First posted 26 Feb 2021cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Safety in reinforcement learning (RL) is a key property in both training and execution in many domains such as autonomous driving or finance. In this paper, we formalize it with a constrained RL formulation in the distributional RL setting. Our general model accepts various definitions of safety(e.g., bounds on expected performance, CVaR, variance, or probability of reaching bad states). To ensure safety during learning, we extend a safe policy optimization method to solve our problem. The distributional RL perspective leads to a more efficient algorithm while additionally catering for natural safe constraints. We empirically validate our propositions on artificial and real domains against appropriate state-of-the-art safe RL algorithms.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 26 Feb 2021.

Builds on

22 of 40 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  5. 2017Safe Reinforcement Learning via Shieldingcited by 427
  6. 2018Reward Constrained Policy Optimizationcited by 397
  7. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  8. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  9. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  10. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  11. 2018Safe Exploration in Continuous Action Spacescited by 237
  12. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
10 more
  1. 2020Projection-Based Constrained Policy Optimizationcited by 195
  2. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  3. 2019IPO: Interior-point Policy Optimization under Constraintscited by 119
  4. 2014Algorithms for CVaR Optimization in MDPscited by 98
  5. 2019Fully Parameterized Quantile Function for Distributional Reinforcement Learningcited by 87
  6. 2016Safe Exploration in Finite Markov Decision Processes with Gaussian Processescited by 72
  7. 2019Reinforcement Learning with Convex Constraintscited by 57
  8. 2019Convergent Policy Optimization for Safe Reinforcement Learningcited by 55
  9. 2014Variance-Constrained Actor-Critic Algorithms for Discounted and Average Reward MDPscited by 28
  10. 2020Reinforcement Learning of Risk-Constrained Policies in Markov Decision Processescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 5; 2024: 2; 2025: 0; 2026: 1.

10 papers.

  1. 2023Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraintscited by 20
  2. 2024A Distributional Analogue to the Successor Representationcited by 15
  3. 2022Robust Reinforcement Learning with Distributional Risk-averse formulationcited by 9
  4. 2024Spectral-Risk Safe Reinforcement Learning with Convergence Guaranteescited by 5
  5. 2021ACReL: Adversarial Conditional value-at-risk Reinforcement Learningcited by 2
  6. 2023Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learningcited by 2
  7. 2023Learning Rewards to Optimize Global Performance Metrics in Deep Reinforcement Learningcited by 1
  8. 2023Modeling Risk in Reinforcement Learning: A Literature Mappingcited by 1
  9. 2023Policy Evaluation in Distributional LQRcited by 0
  10. 2026Safe on Average, Unsafe in the Tail: When Is the Episodic-Cost Tail Controllable?cited by 0