# Latent Gaussian Model Boosting

arXiv: https://arxiv.org/abs/2105.08966 · HTML: https://mlsanity.com/paper/2105.08966/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 57 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 15 of 15)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [To understand deep learning we need to understand kernel learning](https://mlsanity.com/paper/1802.01396/) (2018)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers](https://mlsanity.com/paper/1504.07676/) (2015)
- [Gaussian Process Boosting](https://mlsanity.com/paper/2004.02653/) (2020)
- [Gradient and Newton Boosting for Classification and Regression](https://mlsanity.com/paper/1808.03064/) (2018)
- [Cross-Validation for Correlated Data](https://mlsanity.com/paper/1904.02438/) (2019)
- [Accelerated Gradient Boosting](https://mlsanity.com/paper/1803.02042/) (2018)
- [KTBoost: Combined Kernel and Tree Boosting](https://mlsanity.com/paper/1902.03999/) (2019)
- [Gradient Boosting for Linear Mixed Models](https://mlsanity.com/paper/2011.00947/) (2020)
- [Trees-Based Models for Correlated Data](https://mlsanity.com/paper/2102.08114/) (2021)
- [Accelerating Gradient Boosting Machine](https://mlsanity.com/paper/1903.08708/) (2019)

## Built on by (most cited first, 9 of 9)

- [Machine Learning with High-Cardinality Categorical Features in Actuarial Applications](https://mlsanity.com/paper/2301.12710/) (2023)
- [Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models](https://mlsanity.com/paper/2310.12000/) (2023)
- [Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data](https://mlsanity.com/paper/2405.14492/) (2024)
- [A Comparison of Machine Learning Methods for Data with High-Cardinality Categorical Variables](https://mlsanity.com/paper/2307.02071/) (2023)
- [A Spatio-Temporal Machine Learning Model for Mortgage Credit Risk: Default Probabilities and Loan Portfolios](https://mlsanity.com/paper/2410.02846/) (2024)
- [MFAI: A Scalable Bayesian Matrix Factorization Approach to Leveraging Auxiliary Information](https://mlsanity.com/paper/2303.02566/) (2023)
- [Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects](https://mlsanity.com/paper/2505.09552/) (2025)
- [Vecchia-Inducing-Points Full-Scale Approximations for Gaussian Processes](https://mlsanity.com/paper/2507.05064/) (2025)
- [Spectrally Deconfounded Gradient Boosting](https://mlsanity.com/paper/2607.09371/) (2026)

## Abstract (arXiv)

Latent Gaussian models and boosting are widely used techniques in statistics and machine learning. Tree-boosting shows excellent prediction accuracy on many data sets, but potential drawbacks are that it assumes conditional independence of samples, produces discontinuous predictions for, e.g., spatial data, and it can have difficulty with high-cardinality categorical variables. Latent Gaussian models, such as Gaussian process and grouped random effects models, are flexible prior models which explicitly model dependence among samples and which allow for efficient learning of predictor functions and for making probabilistic predictions. However, existing latent Gaussian models usually assume either a zero or a linear prior mean function which can be an unrealistic assumption. This article introduces a novel approach that combines boosting and latent Gaussian models to remedy the above-mentioned drawbacks and to leverage the advantages of both techniques. We obtain increased prediction accuracy compared to existing approaches in both simulated and real-world data experiments.