mlsanityData to 7 Oct 2026

Adapting the Linearised Laplace Model Evidence for Modern Deep Learning

arXiv 2206.08900 ↗First posted 17 Jun 2022stat.ML cs.AI cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

The linearised Laplace method for estimating model uncertainty has received renewed attention in the Bayesian deep learning community. The method provides reliable error bars and admits a closed-form expression for the model evidence, allowing for scalable selection of model hyperparameters. In this work, we examine the assumptions behind this method, particularly in conjunction with model selection. We show that these interact poorly with some now-standard tools of deep learning--stochastic approximation methods and normalisation layers--and make recommendations for how to better adapt this classic method to the modern setting. We provide theoretical support for our recommendations and validate them empirically on MLPs, classic CNNs, residual networks with and without normalisation layers, generative autoencoders and transformers.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 42 of its 71 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

42 of 71 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Layer Normalizationcited by 5,579
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  7. 2018Group Normalizationcited by 1,514
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 2017Deep Image Priorcited by 1,097
  10. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  11. 2015Weight Uncertainty in Neural Networkscited by 554
  12. 2018Deep Learning for Classical Japanese Literaturecited by 415
30 more
  1. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  2. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  3. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  4. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  5. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  6. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  7. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  8. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 194
  9. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  10. 2019Practical Deep Learning with Bayesian Principlescited by 158
  11. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 139
  12. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  13. 2017L2 Regularization versus Batch and Weight Normalizationcited by 121
  14. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  15. 2020Getting a CLUE: A Method for Explaining Uncertainty Estimatescited by 92
  16. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  17. 2021Characterizing signal propagation to close the performance gap in unnormalized ResNetscited by 87
  18. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 85
  19. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 79
  20. 2018Norm matters: efficient and accurate normalization schemes in deep networkscited by 62
  21. 2020Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Ratecited by 59
  22. 2020Depth Uncertainty in Neural Networkscited by 56
  23. 2020Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisitedcited by 50
  24. 2020Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processescited by 43
  25. 2015Black-box α-divergence Minimizationcited by 37
  26. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
  27. 2018Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learningcited by 35
  28. 2021Fast Adaptation with Linearized Neural Networkscited by 32
  29. 2021On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decaycited by 20
  30. 2019Disentangling and Learning Robust Representations with Natural Clusteringcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 11; 2024: 5; 2025: 6; 2026: 3.

29 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
  3. 2024Bayesian Reward Models for LLM Alignmentcited by 35
  4. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  5. 2023Sampling from Gaussian Process Posteriors using Stochastic Gradient Descentcited by 27
  6. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  7. 2023Fixing Overconfidence in Dynamic Neural Networkscited by 19
  8. 2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18
  9. 2024Reparameterization invariance in approximate Bayesian inferencecited by 17
  10. 2023The Geometry of Neural Nets' Parameter Spaces Under Reparametrizationcited by 16
  11. 2023Improving Neural Additive Models with Bayesian Principlescited by 15
  12. 2023Riemannian Laplace approximations for Bayesian neural networkscited by 14
17 more
  1. 2023Stochastic Marginal Likelihood Gradients using Neural Tangent Kernelscited by 10
  2. 2022Bayesian Experimental Design for Computed Tomography with the Linearised Deep Image Priorcited by 7
  3. 2022Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernelcited by 7
  4. 2024Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihoodcited by 7
  5. 2022Cold Posteriors through PAC-Bayescited by 3
  6. 2023Online Laplace Model Selection Revisitedcited by 2
  7. 2024Active Few-Shot Fine-Tuningcited by 2
  8. 2025Uncertainty-Aware Adaptation of Large Language Models for Protein-Protein Interaction Analysiscited by 2
  9. 2025CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regressioncited by 2
  10. 2023Image Reconstruction via Deep Image Prior Subspacescited by 1
  11. 2025Pointwise confidence estimation in the non-linear ℓ2-regularized least squarescited by 1
  12. 2025Scaling Laws for Uncertainty in Deep Learningcited by 1
  13. 2025Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influencecited by 1
  14. 2026Laplace Approximation for Bayesian Tensor Network Kernel Machinescited by 1
  15. 2025Tubular Riemannian Laplace Approximations for Bayesian Neural Networkscited by 0
  16. 2026Bayesian Fine-tuning in Projected Subspacescited by 0
  17. 2026Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantificationcited by 0