# Predictive Entropy Search for Efficient Global Optimization of Black-box Functions

arXiv: https://arxiv.org/abs/1406.2541 · HTML: https://mlsanity.com/paper/1406.2541/

## Key facts

- Cited by 204 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2014.

## Built on by (most cited first, 30 of 204)

- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement](https://mlsanity.com/paper/2105.08195/) (2021)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [GFlowNets for AI-Driven Scientific Discovery](https://mlsanity.com/paper/2302.00615/) (2023)
- [Prediction-Oriented Bayesian Active Learning](https://mlsanity.com/paper/2304.08151/) (2023)
- [Joint Entropy Search for Maximally-Informed Bayesian Optimization](https://mlsanity.com/paper/2206.04771/) (2022)
- [Pareto Set Learning for Expensive Multi-Objective Optimization](https://mlsanity.com/paper/2210.08495/) (2022)
- [GIBBON: General-purpose Information-Based Bayesian OptimisatioN](https://mlsanity.com/paper/2102.03324/) (2021)
- [Joint Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/2210.02905/) (2022)
- [GAUCHE: A Library for Gaussian Processes in Chemistry](https://mlsanity.com/paper/2212.04450/) (2022)
- [Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information](https://mlsanity.com/paper/2104.09460/) (2021)
- [Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisation](https://mlsanity.com/paper/2301.10123/) (2023)
- [Spiking Graph Convolutional Networks](https://mlsanity.com/paper/2205.02767/) (2022)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent](https://mlsanity.com/paper/2306.11589/) (2023)
- [BORE: Bayesian Optimization by Density-Ratio Estimation](https://mlsanity.com/paper/2102.09009/) (2021)
- [Thinking inside the box: A tutorial on grey-box Bayesian optimization](https://mlsanity.com/paper/2201.00272/) (2022)
- [Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound](https://mlsanity.com/paper/2102.09788/) (2021)
- [Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization](https://mlsanity.com/paper/2106.04335/) (2021)
- [Multi-objective Asynchronous Successive Halving](https://mlsanity.com/paper/2106.12639/) (2021)
- [qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization](https://mlsanity.com/paper/2303.15746/) (2023)
- [Amortized Probabilistic Conditioning for Optimization, Simulation and Inference](https://mlsanity.com/paper/2410.15320/) (2024)
- [Self-Correcting Bayesian Optimization through Bayesian Active Learning](https://mlsanity.com/paper/2304.11005/) (2023)
- [A General Framework for User-Guided Bayesian Optimization](https://mlsanity.com/paper/2311.14645/) (2023)
- [Multi-Fidelity Cost-Aware Bayesian Optimization](https://mlsanity.com/paper/2211.02732/) (2022)
- [A Nonmyopic Approach to Cost-Constrained Bayesian Optimization](https://mlsanity.com/paper/2106.06079/) (2021)
- [A General Recipe for Likelihood-free Bayesian Optimization](https://mlsanity.com/paper/2206.13035/) (2022)

## Abstract (arXiv)

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of the expected reduction in the differential entropy of the predictive distribution. This reformulation allows PES to obtain approximations that are both more accurate and efficient than other alternatives such as Entropy Search (ES). Furthermore, PES can easily perform a fully Bayesian treatment of the model hyperparameters while ES cannot. We evaluate PES in both synthetic and real-world applications, including optimization problems in machine learning, finance, biotechnology, and robotics. We show that the increased accuracy of PES leads to significant gains in optimization performance.