# Spectrally Deconfounded Gradient Boosting

arXiv: https://arxiv.org/abs/2607.09371 · HTML: https://mlsanity.com/paper/2607.09371/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Gaussian Process Boosting](https://mlsanity.com/paper/2004.02653/) (2020)
- [Vecchia approximations of Gaussian-process predictions](https://mlsanity.com/paper/1805.03309/) (2018)
- [Spectral Deconfounding via Perturbed Sparse Linear Models](https://mlsanity.com/paper/1811.05352/) (2018)
- [Doubly Debiased Lasso: High-Dimensional Inference under Hidden Confounding](https://mlsanity.com/paper/2004.03758/) (2020)
- [Latent Gaussian Model Boosting](https://mlsanity.com/paper/2105.08966/) (2021)
- [A lava attack on the recovery of sums of dense and sparse signals](https://mlsanity.com/paper/1502.03155/) (2015)
- [Na"ive regression requires weaker assumptions than factor models to adjust for multiple cause confounding](https://mlsanity.com/paper/2007.12702/) (2020)
- [Spectral Deconfounding for High-Dimensional Sparse Additive Models](https://mlsanity.com/paper/2312.02860/) (2023)
- [Latent confounding in high-dimensional nonlinear models](https://mlsanity.com/paper/2508.06274/) (2025)
- [Laplace Approximations for Mixed-Effects and Gaussian Process Quantile Regression](https://mlsanity.com/paper/2605.21197/) (2026)

## Abstract (arXiv)

Flexible machine-learning methods can be sensitive to hidden confounding: they may learn associations induced by unobserved confounders rather than stable signals. Spectral deconfounding mitigates this problem by shrinking high-variance directions of the covariate matrix that, under dense confounding, carry latent confounder information. Existing work has largely focused on linear models. We develop a nonlinear spectral deconfounding framework for gradient boosting. Our approach replaces the ordinary squared-error loss by a spectral loss, which alters the boosting dynamics by slowing down learning in confounding-aligned directions. We show that deconfounding is not achieved by the spectral loss alone, but by the interaction between spectral shrinkage and regularization, especially in terms of early stopping. Moreover, we provide a mixed-model interpretation that connects LAVA-type shrinkage to random-effects adjustment and yields an empirical-Bayes procedure for tuning the spectral loss. We also extend the method to general likelihoods and nonlinear confounding using Laplace approximations and kernel random effects. Across synthetic and real-world experiments, spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is substantially more scalable than existing nonlinear spectral deconfounding baselines.