mlsanityData to 7 Oct 2026

Riemannian Laplace approximations for Bayesian neural networks

arXiv 2306.07158 ↗First posted 12 Jun 2023stat.ML cs.LG stat.MEcited by 14

Not on the map. Why?

Abstract (arXiv)

Bayesian neural networks often approximate the weight-posterior with a Gaussian distribution. However, practical posteriors are often, even locally, highly non-Gaussian, and empirical performance deteriorates. We propose a simple parametric approximate posterior that adapts to the shape of the true posterior through a Riemannian metric that is determined by the log-posterior gradient. We develop a Riemannian Laplace approximation where samples naturally fall into weight-regions with low negative log-posterior. We show that these samples can be drawn by solving a system of ordinary differential equations, which can be done efficiently by leveraging the structure of the Riemannian metric and automatic differentiation. Empirically, we demonstrate that our approach consistently improves over the conventional Laplace approximation across tasks. We further show that, unlike the conventional Laplace approximation, our method is not overly sensitive to the choice of prior, which alleviates a practical pitfall of current approaches.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 12 Jun 2023.

Builds on

30 of 53 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  6. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  7. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  8. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  9. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  10. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  11. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  12. 2016Eigenvalues of the Hessian in Deep Learning: Singularity and Beyondcited by 156
18 more
  1. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 139
  2. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  3. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  4. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  5. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 85
  6. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 79
  7. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  8. 2019Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningcited by 37
  9. 2016Variational Boosting: Iteratively Refining Posterior Approximationscited by 36
  10. 2022Adapting the Linearised Laplace Model Evidence for Modern Deep Learningcited by 29
  11. 2014Metrics for Probabilistic Geometriescited by 28
  12. 2019Fast and Robust Shortest Paths on Manifolds Learned from Datacited by 25
  13. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  14. 2016A Locally Adaptive Normal Distributioncited by 23
  15. 2023The Geometry of Neural Nets' Parameter Spaces Under Reparametrizationcited by 16
  16. 2022Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networkscited by 14
  17. 2013Probabilistic Solutions to Differential Equations and their Application to Riemannian Statisticscited by 11
  18. 2022Lagrangian Manifold Monte Carlo on Monge Patchescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 6; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?cited by 46
  2. 2024Reparameterization invariance in approximate Bayesian inferencecited by 17
  3. 2023The Interpolating Information Criterion for Overparameterized Modelscited by 11
  4. 2023Riemannian Laplace Approximation with the Fisher Metriccited by 7
  5. 2025What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifoldcited by 6
  6. 2025Geometric Gaussian Approximations of Probability Distributionscited by 2
  7. 2025Laplace Sample Information: Data Informativeness Through a Bayesian Lenscited by 1
  8. 2025Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networkscited by 1
  9. 2026Reducing Memorisation in Generative Models via Riemannian Bayesian Inferencecited by 1
  10. 2025From Overfitting to Reliability: Introducing the Hierarchical Approximate Bayesian Neural Networkcited by 0
  11. 2025Tubular Riemannian Laplace Approximations for Bayesian Neural Networkscited by 0
  12. 2026VOLTA: The Surprising Ineffectiveness of Auxiliary Losses for Calibrated Deep Learningcited by 0
2 more
  1. 2026Monte Carlo Stochastic Depth for Uncertainty Estimation in Deep Learningcited by 0
  2. 2026Don't Stop Me Yet: Sampling Loss Minima via Dissipative Riemannian Mechanicscited by 0