# Sparse within Sparse Gaussian Processes using Neighbor Information

arXiv: https://arxiv.org/abs/2011.05041 · HTML: https://mlsanity.com/paper/2011.05041/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2020.

## Built on by (most cited first, 18 of 18)

- [Calibrating Transformers via Sparse Gaussian Processes](https://mlsanity.com/paper/2303.02444/) (2023)
- [Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2106.01121/) (2021)
- [Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes](https://mlsanity.com/paper/2203.01459/) (2022)
- [Variational Nearest Neighbor Gaussian Process](https://mlsanity.com/paper/2202.01694/) (2022)
- [Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes](https://mlsanity.com/paper/2412.20375/) (2024)
- [Scalable Cross Validation Losses for Gaussian Process Models](https://mlsanity.com/paper/2105.11535/) (2021)
- [Input Dependent Sparse Gaussian Processes](https://mlsanity.com/paper/2107.07281/) (2021)
- [Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression](https://mlsanity.com/paper/2306.14731/) (2023)
- [Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations](https://mlsanity.com/paper/2408.06069/) (2024)
- [Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processes](https://mlsanity.com/paper/2308.14142/) (2023)
- [Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling](https://mlsanity.com/paper/2505.16481/) (2025)
- [B'ezier Gaussian Processes for Tall and Wide Data](https://mlsanity.com/paper/2209.00343/) (2022)
- [Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference](https://mlsanity.com/paper/2407.02476/) (2024)
- [Amortized Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/2409.12301/) (2024)
- [New Bounds for Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2502.08730/) (2025)
- [Further Understanding of a Local Gaussian Process Approximation: Characterising Convergence in the Finite Regime](https://mlsanity.com/paper/2404.06200/) (2024)
- [Sparse Gaussian Processes: Structured Approximations and Power-EP Revisited](https://mlsanity.com/paper/2507.02377/) (2025)
- [Transformed Latent Variable Multi-Output Gaussian Processes](https://mlsanity.com/paper/2605.05133/) (2026)

## Abstract (arXiv)

Approximations to Gaussian processes based on inducing variables, combined with variational inference techniques, enable state-of-the-art sparse approaches to infer GPs at scale through mini batch-based learning. In this work, we address one limitation of sparse GPs, which is due to the challenge in dealing with a large number of inducing variables without imposing a special structure on the inducing inputs. In particular, we introduce a novel hierarchical prior, which imposes sparsity on the set of inducing variables. We treat our model variationally, and we experimentally show considerable computational gains compared to standard sparse GPs when sparsity on the inducing variables is realized considering the nearest inducing inputs of a random mini-batch of the data. We perform an extensive experimental validation that demonstrates the effectiveness of our approach compared to the state-of-the-art. Our approach enables the possibility to use sparse GPs using a large number of inducing points without incurring a prohibitive computational cost.