mlsanityData to 7 Oct 2026

BORE: Bayesian Optimization by Density-Ratio Estimation

arXiv 2102.09009 ↗First posted 17 Feb 2021cs.LG stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. BO proposes solutions according to an explore-exploit trade-off criterion encoded in an acquisition function, many of which are computed from the posterior predictive of a probabilistic surrogate model. Prevalent among these is the expected improvement (EI) function. The need to ensure analytical tractability of the predictive often poses limitations that can hinder the efficiency and applicability of BO. In this paper, we cast the computation of EI as a binary classification problem, building on the link between class-probability estimation and density-ratio estimation, and the lesser-known link between density-ratios and EI. By circumventing the tractability constraints, this reformulation provides numerous advantages, not least in terms of expressiveness, versatility, and scalability.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 30 of its 95 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

30 of 95 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  7. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  8. 2018A Tutorial on Bayesian Optimizationcited by 734
  9. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  10. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  11. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  12. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
18 more
  1. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  2. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  3. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
  4. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  5. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
  6. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
  7. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  8. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
  9. 2015Predictive Entropy Search for Multi-objective Bayesian Optimizationcited by 88
  10. 2016Likelihood-free inference by ratio estimationcited by 80
  11. 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 66
  12. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  13. 2019Tabular Benchmarks for Joint Architecture and Hyperparameter Optimizationcited by 57
  14. 2017Optimization-Based Autonomous Racing of 1:43 Scale RC Carscited by 47
  15. 2018Efficient Bayesian Experimental Design for Implicit Modelscited by 34
  16. 2019Meta-Surrogate Benchmarking for Hyperparameter Optimizationcited by 16
  17. 2019Learning How to Autonomously Race a Car: a Predictive Control Approachcited by 14
  18. 2020Computing the racing line using Bayesian optimizationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 8; 2024: 3; 2025: 5; 2026: 4.

26 papers.

Sort:

By area:

  1. 2023Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performancecited by 143
  2. 2021Automatic Termination for Hyperparameter Optimizationcited by 21
  3. 2022A General Recipe for Likelihood-free Bayesian Optimizationcited by 19
  4. 2023Optimizing Hyperparameters with Conformal Quantile Regressioncited by 10
  5. 2022KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learningcited by 7
  6. 2024Variational Search Distributionscited by 6
  7. 2022Bayesian Optimization For Multi-Objective Mixed-Variable Problemscited by 5
  8. 2022MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimationcited by 5
  9. 2023Pseudo-Bayesian Optimizationcited by 4
  10. 2023Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filteringcited by 3
  11. 2025carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarkscited by 3
  12. 2023MALIBO: Meta-learning for Likelihood-free Bayesian Optimizationcited by 2
14 more
  1. 2024On the development of a practical Bayesian optimisation algorithm for expensive experiments and simulations with changing environmental conditionscited by 2
  2. 2025Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Designcited by 2
  3. 2025Generative Bayesian Optimization: Generative Models as Acquisition Functionscited by 2
  4. 2023Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimisationcited by 1
  5. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputscited by 1
  6. 2026Kernel-based guarantees for nonlinear parametric models in Bayesian optimizationcited by 1
  7. 2021Searching in the Forest for Local Bayesian Optimizationcited by 0
  8. 2023Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learningcited by 0
  9. 2023Deterministic Langevin Unconstrained Optimization with Normalizing Flowscited by 0
  10. 2024Poisson Process for Bayesian Optimizationcited by 0
  11. 2025Bayesian Optimization by Kernel Regression and Density-based Explorationcited by 0
  12. 2025Informing Acquisition Functions via Foundation Models for Molecular Discoverycited by 0
  13. 2026An Open-Source Training Dataset for Foundation Models for Black-box Optimizationcited by 0
  14. 2026Bandits in Prod: Hyperparameter Optimization at Inference Timecited by 0