# A General Recipe for Likelihood-free Bayesian Optimization

arXiv: https://arxiv.org/abs/2206.13035 · HTML: https://mlsanity.com/paper/2206.13035/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 75 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 25 of 25)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search](https://mlsanity.com/paper/1910.11858/) (2019)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [Maximizing acquisition functions for Bayesian optimization](https://mlsanity.com/paper/1805.10196/) (2018)
- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [Few-Shot Bayesian Optimization with Deep Kernel Surrogates](https://mlsanity.com/paper/2101.07667/) (2021)
- [Multi-Agent Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1907.13220/) (2019)
- [Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization](https://mlsanity.com/paper/1905.04970/) (2019)
- [Bayesian Optimization of Composite Functions](https://mlsanity.com/paper/1906.01537/) (2019)
- [Efficient Bayesian Experimental Design for Implicit Models](https://mlsanity.com/paper/1810.09912/) (2018)
- [Bayesian Optimization of Function Networks](https://mlsanity.com/paper/2112.15311/) (2021)
- [Bayesian Optimization with High-Dimensional Outputs](https://mlsanity.com/paper/2106.12997/) (2021)
- [Training Deep Energy-Based Models with f-Divergence Minimization](https://mlsanity.com/paper/2003.03463/) (2020)
- [BORE: Bayesian Optimization by Density-Ratio Estimation](https://mlsanity.com/paper/2102.09009/) (2021)
- [Thinking inside the box: A tutorial on grey-box Bayesian optimization](https://mlsanity.com/paper/2201.00272/) (2022)
- [Generative Adversarial Nets from a Density Ratio Estimation Perspective](https://mlsanity.com/paper/1610.02920/) (2016)
- [Bridging the Gap Between f-GANs and Wasserstein GANs](https://mlsanity.com/paper/1910.09779/) (2019)
- [A Scalable Gradient-Free Method for Bayesian Experimental Design with Implicit Models](https://mlsanity.com/paper/2103.08026/) (2021)
- [A Data-Driven Bayesian Nonparametric Approach for Black-Box Optimization](https://mlsanity.com/paper/2008.02154/) (2020)

## Built on by (most cited first, 19 of 19)

- [Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance](https://mlsanity.com/paper/2304.11127/) (2023)
- [DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Design](https://mlsanity.com/paper/2312.04064/) (2023)
- [Batch Bayesian optimisation via density-ratio estimation with guarantees](https://mlsanity.com/paper/2209.10715/) (2022)
- [Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator](https://mlsanity.com/paper/2212.06751/) (2022)
- [Variational Search Distributions](https://mlsanity.com/paper/2409.06142/) (2024)
- [Pseudo-Bayesian Optimization](https://mlsanity.com/paper/2310.09766/) (2023)
- [A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Lifting](https://mlsanity.com/paper/2404.12219/) (2024)
- [Generative Bayesian Optimization: Generative Models as Acquisition Functions](https://mlsanity.com/paper/2510.25240/) (2025)
- [MALIBO: Meta-learning for Likelihood-free Bayesian Optimization](https://mlsanity.com/paper/2307.03565/) (2023)
- [α-PFN: Fast Entropy Search via In-Context Learning](https://mlsanity.com/paper/2606.07134/) (2026)
- [Conformal Generative Modeling on Triangulated Surfaces](https://mlsanity.com/paper/2303.10251/) (2023)
- [Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputs](https://mlsanity.com/paper/2605.10385/) (2026)
- [Kernel-based guarantees for nonlinear parametric models in Bayesian optimization](https://mlsanity.com/paper/2605.13160/) (2026)
- [Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learning](https://mlsanity.com/paper/2305.15612/) (2023)
- [Poisson Process for Bayesian Optimization](https://mlsanity.com/paper/2402.02687/) (2024)
- [Bayesian Optimization by Kernel Regression and Density-based Exploration](https://mlsanity.com/paper/2502.06178/) (2025)
- [POCAII: Parameter Optimization with Conscious Allocation using Iterative Intelligence](https://mlsanity.com/paper/2505.11745/) (2025)
- [Informing Acquisition Functions via Foundation Models for Molecular Discovery](https://mlsanity.com/paper/2512.13935/) (2025)
- [Bandits in Prod: Hyperparameter Optimization at Inference Time](https://mlsanity.com/paper/2609.01335/) (2026)

## Abstract (arXiv)

The acquisition function, a critical component in Bayesian optimization (BO), can often be written as the expectation of a utility function under a surrogate model. However, to ensure that acquisition functions are tractable to optimize, restrictions must be placed on the surrogate model and utility function. To extend BO to a broader class of models and utilities, we propose likelihood-free BO (LFBO), an approach based on likelihood-free inference. LFBO directly models the acquisition function without having to separately perform inference with a probabilistic surrogate model. We show that computing the acquisition function in LFBO can be reduced to optimizing a weighted classification problem, where the weights correspond to the utility being chosen. By choosing the utility function for expected improvement (EI), LFBO outperforms various state-of-the-art black-box optimization methods on several real-world optimization problems. LFBO can also effectively leverage composite structures of the objective function, which further improves its regret by several orders of magnitude.