mlsanityData to 14 Oct 2026

Efficient Heteroscedastic Bayesian Optimization for Risk-Aware AutoRL

arXiv 2607.26680 ↗First posted 29 Jul 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Reinforcement learning (RL) has shown remarkable success across a wide range of complex tasks. However, RL outcomes can be highly stochastic, and both expected performance and variability often depend on hyperparameter (HP) configurations. We propose efficient and risk-averse heteroscedastic Bayesian Optimization (ERAHBO), a Bayesian optimization method that models both the mean and variance of learning outcomes as functions of the HP configurations. ERAHBO aims to identify HP configurations that achieve high average return while reducing variability across training runs, and it improves the sample efficiency of the HP optimization via adaptive re-sampling rather than a fixed budget per HP. Empirical evaluations across diverse RL algorithms and environments demonstrate that ERAHBO generally outperforms both risk-neutral and risk-averse baselines, delivering improved sample efficiency for risk-averse returns.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 36 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

26 of 36 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  5. 2017Deep Reinforcement Learning that Matterscited by 754
  6. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  7. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 230
  8. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 145
  9. 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 93
  10. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 92
  11. 2015Safe Controller Optimization for Quadrotors with Gaussian Processescited by 78
  12. 2018Information Directed Sampling and Bandits with Heteroscedastic Noisecited by 76
14 more
  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  2. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 63
  3. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  4. 2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 59
  5. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 59
  6. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  7. 2016Practical heteroskedastic Gaussian process modeling for large simulation experimentscited by 39
  8. 2017Replication or exploration? Sequential design for stochastic simulation experimentscited by 30
  9. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  10. 2022Bayesian Generational Population-Based Trainingcited by 20
  11. 2021Risk-averse Heteroscedastic Bayesian Optimizationcited by 17
  12. 2024The Vizier Gaussian Process Bandit Algorithmcited by 14
  13. 2023Batch Bayesian Optimization for Replicable Experimental Designcited by 8
  14. 2024ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Bayesian Dynamic Programming for On-policy Risk-sensitive Reinforcement Learning
  2. 2024EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimization
  3. 2026Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief
  4. 2025Minimax Optimal Reinforcement Learning with Quasi-Optimism
  5. 2025Online Bayesian Risk-Averse Reinforcement Learning
  6. 2026Minimax-Optimality of Posterior Sampling for Reinforcement Learning
  7. 2025Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning
  8. 2025Optimistic Exploration for Risk-Averse Constrained Reinforcement Learning