# MuyGPs: Scalable Gaussian Process Hyperparameter Estimation Using Local Cross-Validation

arXiv: https://arxiv.org/abs/2104.14581 · HTML: https://mlsanity.com/paper/2104.14581/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 37 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [Local Gaussian process approximation for large computer experiments](https://mlsanity.com/paper/1303.0383/) (2013)
- [A Case Study Competition Among Methods for Analyzing Large Spatial Data](https://mlsanity.com/paper/1710.05013/) (2017)
- [The Variational Gaussian Process](https://mlsanity.com/paper/1511.06499/) (2015)
- [A multi-resolution approximation for massive spatial datasets](https://mlsanity.com/paper/1507.04789/) (2015)
- [Multi-resolution filters for massive spatio-temporal data](https://mlsanity.com/paper/1810.04200/) (2018)
- [Precision Aggregated Local Models](https://mlsanity.com/paper/2005.13375/) (2020)
- [Spectral Density Estimation for Random Fields via Periodic Embeddings](https://mlsanity.com/paper/1710.08978/) (2017)

## Built on by (most cited first, 8 of 8)

- [GPLaSDI: Gaussian Process-based Interpretable Latent Space Dynamics Identification through Deep Autoencoder](https://mlsanity.com/paper/2308.05882/) (2023)
- [Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN](https://mlsanity.com/paper/2406.12909/) (2024)
- [Closely-Spaced Object Classification Using MuyGPyS](https://mlsanity.com/paper/2311.10904/) (2023)
- [Fast Gaussian Process Posterior Mean Prediction via Local Cross Validation and Precomputation](https://mlsanity.com/paper/2205.10879/) (2022)
- [Light curve completion and forecasting using fast and scalable Gaussian processes (MuyGPs)](https://mlsanity.com/paper/2208.14592/) (2022)
- [Correspondence of NNGP Kernel and the Matern Kernel](https://mlsanity.com/paper/2410.08311/) (2024)
- [Computationally lightweight classifiers with frequentist bounds on predictions](https://mlsanity.com/paper/2603.22128/) (2026)
- [Scalable Gaussian Process Hyperparameter Optimization via Coverage Regularization](https://mlsanity.com/paper/2209.11280/) (2022)

## Abstract (arXiv)

Gaussian processes (GPs) are non-linear probabilistic models popular in many applications. However, na"ive GP realizations require quadratic memory to store the covariance matrix and cubic computation to perform inference or evaluate the likelihood function. These bottlenecks have driven much investment in the development of approximate GP alternatives that scale to the large data sizes common in modern data-driven applications. We present in this manuscript MuyGPs, a novel efficient GP hyperparameter estimation method. MuyGPs builds upon prior methods that take advantage of the nearest neighbors structure of the data, and uses leave-one-out cross-validation to optimize covariance (kernel) hyperparameters without realizing a possibly expensive likelihood. We describe our model and methods in detail, and compare our implementations against the state-of-the-art competitors in a benchmark spatial statistics problem. We show that our method outperforms all known competitors both in terms of time-to-solution and the root mean squared error of the predictions.