# Additive Gaussian Processes Revisited

arXiv: https://arxiv.org/abs/2206.09861 · HTML: https://mlsanity.com/paper/2206.09861/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 20 Jun 2022.

## Builds on (most cited first, 7 of 7)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Rates of Convergence for Sparse Variational Gaussian Process Regression](https://mlsanity.com/paper/1903.03571/) (2019)
- [Sparse Gaussian Processes with Spherical Harmonic Features](https://mlsanity.com/paper/2006.16649/) (2020)
- [Orthogonal Gaussian process models](https://mlsanity.com/paper/1611.00203/) (2016)
- [lgpr: An interpretable nonparametric method for inferring covariate effects from longitudinal data](https://mlsanity.com/paper/1912.03549/) (2019)

## Built on by (most cited first, 19 of 19)

- [Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?](https://mlsanity.com/paper/2301.12844/) (2023)
- [Improving Neural Additive Models with Bayesian Principles](https://mlsanity.com/paper/2305.16905/) (2023)
- [Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2305.19838/) (2023)
- [High Dimensional Bayesian Optimization using Lasso Variable Selection](https://mlsanity.com/paper/2504.01743/) (2025)
- [Hierarchical shrinkage Gaussian processes: applications to computer code emulation and dynamical system recovery](https://mlsanity.com/paper/2302.00755/) (2023)
- [Tensor Network-Constrained Kernel Machines as Gaussian Processes](https://mlsanity.com/paper/2403.19500/) (2024)
- [Challenges in interpretability of additive models](https://mlsanity.com/paper/2504.10169/) (2025)
- [Automated Model Discovery via Multi-modal & Multi-step Pipeline](https://mlsanity.com/paper/2509.25946/) (2025)
- [Learning Power Flow with Confidence: A Probabilistic Guarantee Framework for Voltage Risk](https://mlsanity.com/paper/2308.07867/) (2023)
- [Trigonometric Quadrature Fourier Features for Scalable Gaussian Process Regression](https://mlsanity.com/paper/2310.14544/) (2023)
- [Beyond Intuition, a Framework for Applying GPs to Real-World Data](https://mlsanity.com/paper/2307.03093/) (2023)
- [Combining additivity and active subspaces for high-dimensional Gaussian process modeling](https://mlsanity.com/paper/2402.03809/) (2024)
- [Omnipresent Yet Overlooked: Heat Kernels in Combinatorial Bayesian Optimization](https://mlsanity.com/paper/2510.26633/) (2025)
- [Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization](https://mlsanity.com/paper/2605.20249/) (2026)
- [Kernel Multigrid: Accelerate Back-fitting via Sparse Gaussian Process Regression](https://mlsanity.com/paper/2403.13300/) (2024)
- [LucidAtlas: Learning Uncertainty-Aware, Covariate-Disentangled, Individualized Atlas Representations](https://mlsanity.com/paper/2502.08445/) (2025)
- [Effect Decomposition of Functional-Output Computer Experiments via Orthogonal Additive Gaussian Processes](https://mlsanity.com/paper/2506.12701/) (2025)
- [On Thompson Sampling and Bilateral Uncertainty in Additive Bayesian Optimization](https://mlsanity.com/paper/2510.11792/) (2025)
- [Meta Additive Model: Interpretable Sparse Learning With Auto Weighting](https://mlsanity.com/paper/2604.20111/) (2026)

## Abstract (arXiv)

Gaussian Process (GP) models are a class of flexible non-parametric models that have rich representational power. By using a Gaussian process with additive structure, complex responses can be modelled whilst retaining interpretability. Previous work showed that additive Gaussian process models require high-dimensional interaction terms. We propose the orthogonal additive kernel (OAK), which imposes an orthogonality constraint on the additive functions, enabling an identifiable, low-dimensional representation of the functional relationship. We connect the OAK kernel to functional ANOVA decomposition, and show improved convergence rates for sparse computation methods. With only a small number of additive low-dimensional terms, we demonstrate the OAK model achieves similar or better predictive performance compared to black-box models, while retaining interpretability.