# Scaling Gaussian Process Regression with Derivatives

arXiv: https://arxiv.org/abs/1810.12283 · HTML: https://mlsanity.com/paper/1810.12283/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 27 of 27)

- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes](https://mlsanity.com/paper/2304.01294/) (2023)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [High-Dimensional Gaussian Process Inference with Derivatives](https://mlsanity.com/paper/2102.07542/) (2021)
- [Scaling Gaussian Processes with Derivative Information Using Variational Inference](https://mlsanity.com/paper/2107.04061/) (2021)
- [Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation](https://mlsanity.com/paper/2206.08366/) (2022)
- [Algorithmic Differentiation for Automated Modeling of Machine Learned Force Fields](https://mlsanity.com/paper/2208.12104/) (2022)
- [Physics-Informed Variational State-Space Gaussian Processes](https://mlsanity.com/paper/2409.13876/) (2024)
- [Global optimization using random embeddings](https://mlsanity.com/paper/2107.12102/) (2021)
- [Using Distance Correlation for Efficient Bayesian Optimization](https://mlsanity.com/paper/2102.08993/) (2021)
- [PAGP: A physics-assisted Gaussian process framework with active learning for forward and inverse problems of partial differential equations](https://mlsanity.com/paper/2204.02583/) (2022)
- [Gaussian Control Barrier Functions : A Non-Parametric Paradigm to Safety](https://mlsanity.com/paper/2203.15474/) (2022)
- [Active search for Bifurcations](https://mlsanity.com/paper/2406.11141/) (2024)
- [Towards Scalable Bayesian Optimization via Gradient-Informed Bayesian Neural Networks](https://mlsanity.com/paper/2504.10076/) (2025)
- [Two-dimensional RMSD projections for reaction path visualization and validation](https://mlsanity.com/paper/2512.07329/) (2025)
- [Scaling Gaussian Process Regression with Full Derivative Observations](https://mlsanity.com/paper/2505.09134/) (2025)
- [When Bayesian Tensor Completion Meets Multioutput Gaussian Processes: Functional Universality and Rank Learning](https://mlsanity.com/paper/2512.21486/) (2025)
- [CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels](https://mlsanity.com/paper/2101.05147/) (2021)
- [A probabilistic Taylor expansion with Gaussian processes](https://mlsanity.com/paper/2102.00877/) (2021)
- [Integrated Gradient attribution for Gaussian Processes with non-Gaussian likelihoods](https://mlsanity.com/paper/2205.12797/) (2022)
- [Large-scale magnetic field maps using structured kernel interpolation for Gaussian process regression](https://mlsanity.com/paper/2310.16574/) (2023)
- [Batch Active Learning in Gaussian Process Regression using Derivatives](https://mlsanity.com/paper/2408.01861/) (2024)
- [Fast Gaussian Processes under Monotonicity Constraints](https://mlsanity.com/paper/2507.06677/) (2025)
- [Gradient-Enhanced Partitioned Gaussian Processes for Real-Time Quadrotor Dynamics Modeling](https://mlsanity.com/paper/2602.12487/) (2026)
- [Gradient-Informed Bayesian and Interior Point Optimization for Efficient Inverse Design in Nanophotonics](https://mlsanity.com/paper/2602.18148/) (2026)
- [Uncertainty-quantified Pulse Signal Recovery from Facial Video using Regularized Stochastic Interpolants](https://mlsanity.com/paper/2604.10777/) (2026)
- [Scalable Derivative Gaussian Processes via Exact Gradient Reduction](https://mlsanity.com/paper/2606.02909/) (2026)

## Abstract (arXiv)

Gaussian processes (GPs) with derivatives are useful in many applications, including Bayesian optimization, implicit surface reconstruction, and terrain reconstruction. Fitting a GP to function values and derivatives at n points in d dimensions requires linear solves and log determinants with an n(d+1) × n(d+1) positive definite matrix -- leading to prohibitive O(n3d3) computations for standard direct methods. We propose iterative solvers using fast O(nd) matrix-vector multiplications (MVMs), together with pivoted Cholesky preconditioning that cuts the iterations to convergence by several orders of magnitude, allowing for fast kernel learning and prediction. Our approaches, together with dimensionality reduction, enables Bayesian optimization with derivatives to scale to high-dimensional problems and large evaluation budgets.