mlsanityData to 7 Oct 2026

Gradient Boosting Performs Gaussian Process Inference

arXiv 2206.05608 ↗First posted 11 Jun 2022cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper shows that gradient boosting based on symmetric decision trees can be equivalently reformulated as a kernel method that converges to the solution of a certain Kernel Ridge Regression problem. Thus, we obtain the convergence to a Gaussian Process' posterior mean, which, in turn, allows us to easily transform gradient boosting into a sampler from the posterior to provide better knowledge uncertainty estimates through Monte-Carlo estimation of the posterior variance. We show that the proposed sampler allows for better knowledge uncertainty estimates leading to improved out-of-domain detection.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 11 Jun 2022.

Builds on

19 of 35 references.

References · 19

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  6. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  7. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  8. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  9. 2017Deep Neural Networks as Gaussian Processescited by 617
  10. 2016Generalized Random Forestscited by 309
  11. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  12. 2019Ensemble Distribution Distillationcited by 158
7 more
  1. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 146
  2. 2020Uncertainty in Gradient Boosting via Ensemblescited by 56
  3. 2020Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Samplingcited by 32
  4. 2020SGLB: Stochastic Gradient Langevin Boostingcited by 12
  5. 2016The Mondrian Kernelcited by 9
  6. 2019Minimal Variance Sampling in Stochastic Gradient Boostingcited by 5
  7. 2021A Neural Tangent Kernel Perspective of Infinite Tree Ensemblescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Reliable and scalable variable importance estimation via warm-start and early stoppingcited by 2
  2. 2025Statistical Inference for Gradient Boosting Regressioncited by 2
  3. 2024Universal Feature Selection for Simultaneous Interpretability of Multitask Datasetscited by 1
  4. 2026Statistical Inference for Explainable Boosting Machinescited by 0