mlsanityData to 7 Oct 2026

Self-Correcting Bayesian Optimization through Bayesian Active Learning

arXiv 2304.11005 ↗First posted 21 Apr 2023cs.LG stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Gaussian processes are the model of choice in Bayesian optimization and active learning. Yet, they are highly dependent on cleverly chosen hyperparameters to reach their full potential, and little effort is devoted to finding good hyperparameters in the literature. We demonstrate the impact of selecting good hyperparameters for GPs and present two acquisition functions that explicitly prioritize hyperparameter learning. Statistical distance-based Active Learning (SAL) considers the average disagreement between samples from the posterior, as measured by a statistical distance. SAL outperforms the state-of-the-art in Bayesian active learning on several test functions. We then introduce Self-Correcting Bayesian Optimization (SCoreBO), which extends SAL to perform Bayesian optimization and active learning simultaneously. SCoreBO learns the model hyperparameters at improved rates compared to vanilla BO, while outperforming the latest Bayesian optimization methods on traditional benchmarks. Moreover, we demonstrate the importance of self-correction on atypical Bayesian optimization tasks.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 47 of its 79 references that are in the map. First posted on arXiv 21 Apr 2023.

Builds on

47 of 79 references.

References · 47

  1. 2018A Tutorial on Bayesian Optimizationcited by 734
  2. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  3. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  4. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  5. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  6. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  7. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
  8. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  9. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  10. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  11. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  12. 2016Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Roboticscited by 134
35 more
  1. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 125
  2. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  3. 2015Bayesian optimization for materials designcited by 78
  4. 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 69
  5. 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 66
  6. 2019Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variablescited by 64
  7. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  8. 2017Bayesian Optimization with Gradientscited by 59
  9. 2018Bayesian Optimization in AlphaGocited by 55
  10. 2022Joint Entropy Search for Maximally-Informed Bayesian Optimizationcited by 55
  11. 2017Improving the Expected Improvement Algorithmcited by 54
  12. 2021Misspecified Gaussian Process Bandit Optimizationcited by 52
  13. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
  14. 2019No-Regret Bayesian Optimization with Unknown Hyperparameterscited by 50
  15. 2022Constrained Efficient Global Optimization of Expensive Black-box Functionscited by 49
  16. 2021GIBBON: General-purpose Information-Based Bayesian OptimisatioNcited by 47
  17. 2022Joint Entropy Search for Multi-objective Bayesian Optimizationcited by 45
  18. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
  19. 2018Practical Design Space Explorationcited by 35
  20. 2019Approximate Inference for Fully Bayesian Gaussian Process Regressioncited by 31
  21. 2020Convergence Guarantees for Gaussian Process Means With Misspecified Likelihoods and Smoothnesscited by 29
  22. 2020Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriorscited by 29
  23. 2022Local Bayesian optimization via maximizing probability of descentcited by 29
  24. 2017Replication or exploration? Sequential design for stochastic simulation experimentscited by 28
  25. 2022Bayesian Optimization with Conformal Prediction Setscited by 27
  26. 2022Bayesian Active Learning with Fully Bayesian Gaussian Processescited by 26
  27. 2017Constrained Bayesian Optimization for Automatic Chemical Designcited by 20
  28. 2017Fast Information-theoretic Bayesian Optimisationcited by 19
  29. 2023Randomized Gaussian Process Upper Confidence Bound with Tighter Bayesian Regret Boundscited by 16
  30. 2022Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integrationcited by 10
  31. 2015The Configurable SAT Solver Challenge (CSSC)cited by 9
  32. 2022Learning Skill-based Industrial Robot Tasks with User Priorscited by 9
  33. 2018Distance-distributed design for Gaussian process surrogatescited by 7
  34. 2020What do you Mean? The Role of the Mean Function in Bayesian Optimisationcited by 7
  35. 2022Rectified Max-Value Entropy Search for Bayesian Optimizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 6; 2025: 5; 2026: 7.

22 papers.

Sort:

By area:

  1. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  2. 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
  3. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  4. 2024Time-Varying Gaussian Process Bandits with Unknown Priorcited by 10
  5. 2024Robust Gaussian Processes via Relevance Pursuitcited by 7
  6. 2024Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimalcited by 6
  7. 2023High-dimensional Bayesian Optimization with Group Testingcited by 5
  8. 2023Bayesian Active Learning in the Presence of Nuisance Parameterscited by 5
  9. 2024No-Regret Learning of Nash Equilibrium for Black-Box Games via Gaussian Processescited by 3
  10. 2023Provably Efficient Bayesian Optimization with Unknown Gaussian Process Hyperparameter Estimationcited by 2
  11. 2025Wasserstein Barycenter Gaussian Process based Bayesian Optimizationcited by 2
  12. 2025Informed Initialization for Bayesian Optimization and Active Learningcited by 2
10 more
  1. 2025Adaptive Prior Selection in Gaussian Process Bandits with Thompson Samplingcited by 1
  2. 2025Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testingcited by 1
  3. 2026α-PFN: Fast Entropy Search via In-Context Learningcited by 1
  4. 2026Enhancing Bayesian Optimization and Active Learning Through Kernel Diversitycited by 1
  5. 2025Direct Regret Optimization in Bayesian Optimizationcited by 0
  6. 2026Agentic AI for Self-Driving Laboratories in Soft Matter: Taxonomy, Benchmarks,and Open Challengescited by 0
  7. 2026Pragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active Inferencecited by 0
  8. 2026Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization taskscited by 0
  9. 2026Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discoverycited by 0
  10. 2026Bayesian optimization with kernel ensembles and disagreement-based acquisition for source localization and acoustic inversioncited by 0