# Randomly Projected Additive Gaussian Processes for Regression

arXiv: https://arxiv.org/abs/1912.12834 · HTML: https://mlsanity.com/paper/1912.12834/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2019.

## Built on by (most cited first, 19 of 19)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB](https://mlsanity.com/paper/2303.00890/) (2023)
- [Kernel Interpolation for Scalable Online Gaussian Processes](https://mlsanity.com/paper/2103.01454/) (2021)
- [Spatio-Temporal Attention and Gaussian Processes for Personalized Video Gaze Estimation](https://mlsanity.com/paper/2404.05215/) (2024)
- [Scalable Gaussian-process regression and variable selection using Vecchia approximations](https://mlsanity.com/paper/2202.12981/) (2022)
- [High Dimensional Bayesian Optimization with Kernel Principal Component Analysis](https://mlsanity.com/paper/2204.13753/) (2022)
- [Additive Multi-Index Gaussian process modeling, with application to multi-physics surrogate modeling of the quark-gluon plasma](https://mlsanity.com/paper/2306.07299/) (2023)
- [Dynamic Online Ensembles of Basis Expansions](https://mlsanity.com/paper/2405.01365/) (2024)
- [A spectral mixture representation of isotropic kernels with application to random Fourier features](https://mlsanity.com/paper/2411.02770/) (2024)
- [Sensitivity Prewarping for Local Surrogate Modeling](https://mlsanity.com/paper/2101.06296/) (2021)
- [High-dimensional additive Gaussian processes under monotonicity constraints](https://mlsanity.com/paper/2205.08528/) (2022)
- [From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximation](https://mlsanity.com/paper/2502.10540/) (2025)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [Combining additivity and active subspaces for high-dimensional Gaussian process modeling](https://mlsanity.com/paper/2402.03809/) (2024)
- [Representing Additive Gaussian Processes by Sparse Matrices](https://mlsanity.com/paper/2305.00324/) (2023)
- [Kernel Multigrid: Accelerate Back-fitting via Sparse Gaussian Process Regression](https://mlsanity.com/paper/2403.13300/) (2024)
- [GNet: A scalable and flexible Gaussian process network with nonparametric neurons](https://mlsanity.com/paper/2607.10735/) (2026)
- [Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation](https://mlsanity.com/paper/2609.13396/) (2026)

## Abstract (arXiv)

Gaussian processes (GPs) provide flexible distributions over functions, with inductive biases controlled by a kernel. However, in many applications Gaussian processes can struggle with even moderate input dimensionality. Learning a low dimensional projection can help alleviate this curse of dimensionality, but introduces many trainable hyperparameters, which can be cumbersome, especially in the small data regime. We use additive sums of kernels for GP regression, where each kernel operates on a different random projection of its inputs. Surprisingly, we find that as the number of random projections increases, the predictive performance of this approach quickly converges to the performance of a kernel operating on the original full dimensional inputs, over a wide range of data sets, even if we are projecting into a single dimension. As a consequence, many problems can remarkably be reduced to one dimensional input spaces, without learning a transformation. We prove this convergence and its rate, and additionally propose a deterministic approach that converges more quickly than purely random projections. Moreover, we demonstrate our approach can achieve faster inference and improved predictive accuracy for high-dimensional inputs compared to kernels in the original input space.