mlsanityData to 7 Oct 2026

A General Recipe for Likelihood-free Bayesian Optimization

arXiv 2206.13035 ↗First posted 27 Jun 2022cs.LG cs.AI stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

The acquisition function, a critical component in Bayesian optimization (BO), can often be written as the expectation of a utility function under a surrogate model. However, to ensure that acquisition functions are tractable to optimize, restrictions must be placed on the surrogate model and utility function. To extend BO to a broader class of models and utilities, we propose likelihood-free BO (LFBO), an approach based on likelihood-free inference. LFBO directly models the acquisition function without having to separately perform inference with a probabilistic surrogate model. We show that computing the acquisition function in LFBO can be reduced to optimizing a weighted classification problem, where the weights correspond to the utility being chosen. By choosing the utility function for expected improvement (EI), LFBO outperforms various state-of-the-art black-box optimization methods on several real-world optimization problems. LFBO can also effectively leverage composite structures of the objective function, which further improves its regret by several orders of magnitude.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 25 of its 75 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

25 of 75 references.

References · 25

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  5. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  6. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  7. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
  8. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  9. 2016Learning in Implicit Generative Modelscited by 124
  10. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
  11. 2015Predictive Entropy Search for Multi-objective Bayesian Optimizationcited by 88
  12. 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 70
13 more
  1. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  2. 2019Tabular Benchmarks for Joint Architecture and Hyperparameter Optimizationcited by 57
  3. 2019Bayesian Optimization of Composite Functionscited by 45
  4. 2018Efficient Bayesian Experimental Design for Implicit Modelscited by 34
  5. 2021Bayesian Optimization of Function Networkscited by 33
  6. 2021Bayesian Optimization with High-Dimensional Outputscited by 32
  7. 2020Training Deep Energy-Based Models with f-Divergence Minimizationcited by 29
  8. 2021BORE: Bayesian Optimization by Density-Ratio Estimationcited by 26
  9. 2022Thinking inside the box: A tutorial on grey-box Bayesian optimizationcited by 26
  10. 2016Generative Adversarial Nets from a Density Ratio Estimation Perspectivecited by 24
  11. 2019Bridging the Gap Between f-GANs and Wasserstein GANscited by 23
  12. 2021A Scalable Gradient-Free Method for Bayesian Experimental Design with Implicit Modelscited by 7
  13. 2020A Data-Driven Bayesian Nonparametric Approach for Black-Box Optimizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 3; 2025: 4; 2026: 4.

19 papers.

Sort:

By area:

  1. 2023Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performancecited by 143
  2. 2023DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Designcited by 12
  3. 2022Batch Bayesian optimisation via density-ratio estimation with guaranteescited by 9
  4. 2022Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimatorcited by 7
  5. 2024Variational Search Distributionscited by 6
  6. 2023Pseudo-Bayesian Optimizationcited by 4
  7. 2024A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Liftingcited by 3
  8. 2023MALIBO: Meta-learning for Likelihood-free Bayesian Optimizationcited by 2
  9. 2025Generative Bayesian Optimization: Generative Models as Acquisition Functionscited by 2
  10. 2023Conformal Generative Modeling on Triangulated Surfacescited by 1
  11. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputscited by 1
  12. 2026Kernel-based guarantees for nonlinear parametric models in Bayesian optimizationcited by 1
7 more
  1. 2026α-PFN: Fast Entropy Search via In-Context Learningcited by 1
  2. 2023Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learningcited by 0
  3. 2024Poisson Process for Bayesian Optimizationcited by 0
  4. 2025Bayesian Optimization by Kernel Regression and Density-based Explorationcited by 0
  5. 2025POCAII: Parameter Optimization with Conscious Allocation using Iterative Intelligencecited by 0
  6. 2025Informing Acquisition Functions via Foundation Models for Molecular Discoverycited by 0
  7. 2026Bandits in Prod: Hyperparameter Optimization at Inference Timecited by 0