mlsanityData to 7 Oct 2026

Stochastic Variational Deep Kernel Learning

arXiv 1611.00336 ↗First posted 1 Nov 2016stat.ML cs.LG stat.MEcited by 92

Not on the map. Why?

Abstract (arXiv)

Deep kernel learning combines the non-parametric flexibility of kernel methods with the inductive biases of deep learning architectures. We propose a novel deep kernel learning model and stochastic variational inference procedure which generalizes deep kernel learning approaches to enable classification, multi-task learning, additive covariance structures, and stochastic gradient training. Specifically, we apply additive base kernels to subsets of output features from deep neural architectures, and jointly learn the parameters of the base kernels and deep network through a Gaussian process marginal likelihood objective. Within this framework, we derive an efficient form of stochastic variational inference which leverages local kernel interpolation, inducing points, and structure exploiting algebra. We show improved performance over stand alone deep networks, SVMs, and state of the art scalable Gaussian processes on several classification benchmarks, including an airline delay dataset containing 6 million training points, CIFAR, and ImageNet.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 20; 2023: 12; 2024: 14; 2025: 13; 2026: 7.

92 papers.

Sort:

By area:

  1. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  2. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
  3. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  4. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  5. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  6. 2021Personalized Federated Learning with Gaussian Processescited by 76
  7. 2024Variational Bayesian Last Layerscited by 67
  8. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  9. 2021Causal Inference for Time series Analysis: Problems, Methods and Evaluationcited by 42
  10. 2021Meta-Cal: Well-controlled Post-hoc Calibration by Rankingcited by 33
  11. 2024Diffusion Models as Constrained Samplers for Optimization with Unknown Constraintscited by 33
  12. 2021Kernel Continual Learningcited by 27
38 more
  1. 2021GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learningcited by 26
  2. 2022Integrating Random Effects in Deep Neural Networkscited by 22
  3. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  4. 2022Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Predictioncited by 21
  5. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  6. 2024Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inferencecited by 16
  7. 2021Deep Gaussian Processes: A Surveycited by 13
  8. 2024Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learningcited by 11
  9. 2023Incorporating Unlabelled Data into Bayesian Neural Networkscited by 9
  10. 2021Toward a `Standard Model' of Machine Learningcited by 8
  11. 2022Deep Variational Implicit Processescited by 8
  12. 2024Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noisecited by 8
  13. 2021Improving Robustness and Uncertainty Modelling in Neural Ordinary Differential Equationscited by 7
  14. 2024DKL-KAN: Scalable Deep Kernel Learning using Kolmogorov-Arnold Networkscited by 7
  15. 2021Hierarchical Inducing Point Gaussian Process for Inter-domain Observationscited by 6
  16. 2022Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Datacited by 6
  17. 2023Guided Deep Kernel Learningcited by 6
  18. 2023Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processescited by 6
  19. 2021Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Surveycited by 5
  20. 2021Active Assessment of Prediction Services as Accuracy Surface Over Attribute Combinationscited by 5
  21. 2022Energy-Based Models for Functional Data using Path Measure Tiltingcited by 5
  22. 2022Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Processcited by 5
  23. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
  24. 2021Inferring Black Hole Properties from Astronomical Multivariate Time Series with Bayesian Attentive Neural Processescited by 4
  25. 2021End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimizationcited by 4
  26. 2022Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guaranteescited by 4
  27. 2024Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processescited by 4
  28. 2021Calibrated simplex-mapping classificationcited by 3
  29. 2021GP-ConvCNP: Better Generalization for Convolutional Conditional Neural Processes on Time Series Datacited by 3
  30. 2021Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefitscited by 3
  31. 2022Uncertainty Calibration in Bayesian Neural Networks via Distance-Aware Priorscited by 3
  32. 2022Synthetic data enable experiments in atomistic machine learningcited by 3
  33. 2022Gaussian Process Latent Variable Modeling for Few-shot Time Series Forecastingcited by 3
  34. 2023Convolutional Deep Kernel Machinescited by 3
  35. 2021Uncertainty-aware Remaining Useful Life predictorcited by 2
  36. 2022Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernelcited by 2
  37. 2022Sparse Kernel Gaussian Processes through Iterative Charted Refinement (ICR)cited by 2
  38. 2023Gaussian Process-Gated Hierarchical Mixtures of Expertscited by 2

50 most cited of 92 All 92 (JSON)