# Response to Promises and Pitfalls of Deep Kernel Learning

arXiv: https://arxiv.org/abs/2509.21228 · HTML: https://mlsanity.com/paper/2509.21228/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders](https://mlsanity.com/paper/2203.12742/) (2022)
- [Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/1611.00336/) (2016)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Manifold Gaussian Processes for Regression](https://mlsanity.com/paper/1402.5876/) (2014)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Deep Learning is Not So Mysterious or Different](https://mlsanity.com/paper/2503.02113/) (2025)
- [A Study of Bayesian Neural Network Surrogates for Bayesian Optimization](https://mlsanity.com/paper/2305.20028/) (2023)
- [Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks](https://mlsanity.com/paper/1707.02476/) (2017)
- [Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance](https://mlsanity.com/paper/1805.10407/) (2018)
- [Learning Scalable Deep Kernels with Recurrent Structure](https://mlsanity.com/paper/1610.08936/) (2016)
- [Deep Bayesian Gaussian Processes for Uncertainty Estimation in Electronic Health Records](https://mlsanity.com/paper/2003.10170/) (2020)
- [Physics Informed Deep Kernel Learning](https://mlsanity.com/paper/2006.04976/) (2020)
- [Deep Kernel Survival Analysis and Subject-Specific Survival Time Prediction Intervals](https://mlsanity.com/paper/2007.12975/) (2020)
- [Few-shot Adaptation for Manipulating Granular Materials Under Domain Shift](https://mlsanity.com/paper/2303.02893/) (2023)

## Abstract (arXiv)

This note responds to "Promises and Pitfalls of Deep Kernel Learning" (Ober et al., 2021). The marginal likelihood of a Gaussian process can be compartmentalized into a data fit term and a complexity penalty. Ober et al. (2021) shows that if a kernel can be multiplied by a signal variance coefficient, then reparametrizing and substituting in the maximized value of this parameter sets a reparametrized data fit term to a fixed value. They use this finding to argue that the complexity penalty, a log determinant of the kernel matrix, then dominates in determining the other values of kernel hyperparameters, which can lead to data overcorrelation. By contrast, we show that the reparametrization in fact introduces another data-fit term which influences all other kernel hyperparameters. Thus, a balance between data fit and complexity still plays a significant role in determining kernel hyperparameters.