mlsanityData to 7 Oct 2026

Risk-Averse Bayes-Adaptive Reinforcement Learning

arXiv 2102.05762 ↗First posted 10 Feb 2021cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

In this work, we address risk-averse Bayes-adaptive reinforcement learning. We pose the problem of optimising the conditional value at risk (CVaR) of the total return in Bayes-adaptive Markov decision processes (MDPs). We show that a policy optimising CVaR in this setting is risk-averse to both the parametric uncertainty due to the prior distribution over MDPs, and the internal uncertainty due to the inherent stochasticity of MDPs. We reformulate the problem as a two-player stochastic game and propose an approximate algorithm based on Monte Carlo tree search and Bayesian optimisation. Our experiments demonstrate that our approach significantly outperforms baseline approaches for this problem.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 16 of its 56 references that are in the map. First posted on arXiv 10 Feb 2021.

Builds on

16 of 56 references.

References · 16

  1. 2017Robust Adversarial Reinforcement Learningcited by 439
  2. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  3. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  4. 2014Optimizing the CVaR via Samplingcited by 98
  5. 2014Algorithms for CVaR Optimization in MDPscited by 98
  6. 2017The Uncertainty Bellman Equation and Explorationcited by 81
  7. 2015Policy Gradient for Coherent Risk Measurescited by 74
  8. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
  9. 2019Being Optimistic to Be Conservative: Quickly Learning a CVaR Policycited by 63
  10. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 54
  11. 2019Worst Cases Policy Gradientscited by 52
  12. 2019A Bayesian Approach to Robust Reinforcement Learningcited by 32
4 more
  1. 2014Policy Gradients for CVaR-Constrained MDPscited by 20
  2. 2020Bayesian Optimized Monte Carlo Planningcited by 16
  3. 2019Robust and Adaptive Planning under Model Uncertaintycited by 10
  4. 2020Minimax Regret Optimisation for Robust Planning in Uncertain Markov Decision Processescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 8; 2024: 4; 2025: 6; 2026: 2.

28 papers.

Sort:

By area:

  1. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 40
  2. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  3. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
  4. 2024Robust Markov Decision Processes: A Place Where AI and Formal Methods Meetcited by 18
  5. 2023On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processescited by 17
  6. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  7. 2021Two steps to risk sensitivitycited by 8
  8. 2023Decision-Making Under Uncertainty: Beyond Probabilitiescited by 8
  9. 2024Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planningcited by 7
  10. 2023Learning to Forecast Aleatoric and Epistemic Uncertainties over Long Horizon Trajectoriescited by 6
  11. 2021Planning for Risk-Aversion and Expected Value in MDPscited by 5
  12. 2022Risk-Sensitive Bayesian Games for Multi-Agent Reinforcement Learning under Policy Uncertaintycited by 5
16 more
  1. 2025Return Capping: Sample-Efficient CVaR Policy Gradient Optimisationcited by 5
  2. 2021Merging Two Cultures: Deep and Statistical Learningcited by 4
  3. 2023Bayesian Risk-Averse Q-Learning with Streaming Observationscited by 4
  4. 2024Factorised Active Inference for Strategic Multi-Agent Interactionscited by 4
  5. 2021Learning to Be Cautiouscited by 3
  6. 2023Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterioncited by 3
  7. 2023Bayesian inference for data-efficient, explainable, and safe robotic motion planning: A reviewcited by 1
  8. 2025Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Reviewcited by 1
  9. 2025On the Fundamental Limitations of Dual Static CVaR Decompositions in Markov Decision Processescited by 1
  10. 2025Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatismcited by 1
  11. 2023Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planningcited by 0
  12. 2023Adaptive Policy Learning to Additional Taskscited by 0
  13. 2024Offline Bayesian Aleatoric and Epistemic Uncertainty Quantification and Posterior Value Optimisation in Finite-State MDPscited by 0
  14. 2025Risk-averse policies for natural gas futures trading using distributional reinforcement learningcited by 0
  15. 2026Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Beliefcited by 0
  16. 2026Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specificationscited by 0