# Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature

arXiv: https://arxiv.org/abs/1704.00060 · HTML: https://mlsanity.com/paper/1704.00060/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2017.

## Built on by (most cited first, 9 of 9)

- [Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation](https://mlsanity.com/paper/2206.08366/) (2022)
- [Gaussian Process Constraint Learning for Scalable Chance-Constrained Motion Planning from Demonstrations](https://mlsanity.com/paper/2112.04612/) (2021)
- [Bayesian optimization with improved scalability and derivative information for efficient design of nanophotonic structures](https://mlsanity.com/paper/2101.02972/) (2021)
- [Using Distance Correlation for Efficient Bayesian Optimization](https://mlsanity.com/paper/2102.08993/) (2021)
- [Towards Scalable Bayesian Optimization via Gradient-Informed Bayesian Neural Networks](https://mlsanity.com/paper/2504.10076/) (2025)
- [Improving Knowledge Distillation in Transfer Learning with Layer-wise Learning Rates](https://mlsanity.com/paper/2407.04871/) (2024)
- [Curvature-aware Expected Free Energy as an Acquisition Function for Bayesian Optimization](https://mlsanity.com/paper/2603.26339/) (2026)
- [Density reconstruction from schlieren images through Bayesian nonparametric models](https://mlsanity.com/paper/2201.05233/) (2022)
- [Batch Active Learning in Gaussian Process Regression using Derivatives](https://mlsanity.com/paper/2408.01861/) (2024)

## Abstract (arXiv)

An exciting branch of machine learning research focuses on methods for learning, optimizing, and integrating unknown functions that are difficult or costly to evaluate. A popular Bayesian approach to this problem uses a Gaussian process (GP) to construct a posterior distribution over the function of interest given a set of observed measurements, and selects new points to evaluate using the statistics of this posterior. Here we extend these methods to exploit derivative information from the unknown function. We describe methods for Bayesian optimization (BO) and Bayesian quadrature (BQ) in settings where first and second derivatives may be evaluated along with the function itself. We perform sampling-based inference in order to incorporate uncertainty over hyperparameters, and show that both hyperparameter and function uncertainty decrease much more rapidly when using derivative information. Moreover, we introduce techniques for overcoming ill-conditioning issues that have plagued earlier methods for gradient-enhanced Gaussian processes and kriging. We illustrate the efficacy of these methods using applications to real and simulated Bayesian optimization and quadrature problems, and show that exploting derivatives can provide substantial gains over standard methods.