mlsanityData to 7 Oct 2026

Approximate Inference Turns Deep Networks into Gaussian Processes

arXiv 1906.01930 ↗First posted 5 Jun 2019stat.ML cs.AI cs.LGcited by 89

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNN) and Gaussian processes (GP) are two powerful models with several theoretical connections relating them, but the relationship between their training methods is not well understood. In this paper, we show that certain Gaussian posterior approximations for Bayesian DNNs are equivalent to GP posteriors. This enables us to relate solutions and iterations of a deep-learning algorithm to GP inference. As a result, we can obtain a GP kernel and a nonlinear feature map while training a DNN. Surprisingly, the resulting kernel is the neural tangent kernel. We show kernels obtained on real datasets and demonstrate the use of the GP marginal likelihood to tune hyperparameters of DNNs. Our work aims to facilitate further research on combining DNNs and GPs in practical settings.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 18; 2023: 23; 2024: 16; 2025: 11; 2026: 8.

89 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  3. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  4. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  5. 2021The Bayesian Learning Rulecited by 95
  6. 2024Continual Learning and Catastrophic Forgettingcited by 73
  7. 2022Fast Finite Width Neural Tangent Kernelcited by 62
  8. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  9. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  10. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  11. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  12. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
38 more
  1. 2021Fast Adaptation with Linearized Neural Networkscited by 32
  2. 2022Adapting the Linearised Laplace Model Evidence for Modern Deep Learningcited by 29
  3. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  4. 2022NeuralEF: Deconstructing Kernels by Deep Neural Networkscited by 26
  5. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  6. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  7. 2022Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEscited by 24
  8. 2024Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Modelscited by 20
  9. 2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18
  10. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 18
  11. 2023Variational Linearized Laplace Approximation for Bayesian Deep Learningcited by 17
  12. 2024Reparameterization invariance in approximate Bayesian inferencecited by 17
  13. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  14. 2023The Memory Perturbation Equation: Understanding Model's Sensitivity to Datacited by 16
  15. 2023Improving Neural Additive Models with Bayesian Principlescited by 15
  16. 2023Riemannian Laplace approximations for Bayesian neural networkscited by 14
  17. 2021Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processescited by 12
  18. 2021Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guaranteescited by 12
  19. 2024Linearization Turns Neural Operators into Function-Valued Gaussian Processescited by 11
  20. 2024FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learningcited by 11
  21. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  22. 2022Uncertainty Estimation for Computed Tomography with a Linearised Deep Image Priorcited by 10
  23. 2023Promises and Pitfalls of the Linearized Laplace in Bayesian Optimizationcited by 10
  24. 2023Stochastic Marginal Likelihood Gradients using Neural Tangent Kernelscited by 10
  25. 2025Confidence Sequences for Generalized Linear Models via Regret Analysiscited by 9
  26. 2024Active Fine-Tuning of Multi-Task Policiescited by 8
  27. 2021Bridging the Gap Between Target Networks and Functional Regularizationcited by 7
  28. 2022Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernelcited by 7
  29. 2023Probabilistic Circuits That Know What They Don't Knowcited by 7
  30. 2024A prediction rigidity formalism for low-cost uncertainties in trained neural networkscited by 7
  31. 2024Function-space Parameterization of Neural Networks for Sequential Learningcited by 7
  32. 2023Guided Deep Kernel Learningcited by 6
  33. 2023Uncertainty and Structure in Neural Ordinary Differential Equationscited by 6
  34. 2023On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspectivecited by 6
  35. 2025Knowledge Adaptation as Posterior Correctioncited by 6
  36. 2023Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networkscited by 4
  37. 2023On permutation symmetries in Bayesian neural network posteriors: a variational perspectivecited by 4
  38. 2025Connecting Federated ADMM to Bayescited by 4

50 most cited of 89 All 89 (JSON)