mlsanityData to 14 Oct 2026

Humble your Overconfident Networks: Unlearning Overfitting via Sequential Monte Carlo Tempered Deep Ensembles

arXiv 2505.11671 ↗First posted 16 May 2025stat.ML cs.LG stat.COcited by 0

Not on the map. Why?

Abstract (arXiv)

Sequential Monte Carlo (SMC) methods offer a principled approach to Bayesian uncertainty quantification but are traditionally limited by the need for full-batch gradient evaluations. We introduce a scalable variant by incorporating Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) proposals into SMC, enabling efficient mini-batch based sampling. Our resulting SMCSGHMC algorithm outperforms standard stochastic gradient descent (SGD) and deep ensembles across image classification, out-of-distribution (OOD) detection, and transfer learning tasks. We further show that SMCSGHMC mitigates overfitting and improves calibration, providing a flexible, scalable pathway for converting pretrained neural networks into well-calibrated Bayesian models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 40 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

14 of 40 references.

References · 14

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  6. 2016An overview of gradient descent optimization algorithmscited by 1,033
  7. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  8. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  9. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  10. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 113
  11. 2013Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networkscited by 84
  12. 2020A statistical theory of cold posteriors in deep neural networkscited by 54
2 more
  1. 2017Bayesian Recurrent Neural Networkscited by 39
  2. 2020Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Driftscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Enhancing Low-Precision Sampling via Stochastic Gradient Hamiltonian Monte Carlo
  2. 2026Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance
  3. 2026Non-Asymptotic Error Bounds for SMC with Biased Proposals: Application to Conditional Diffusion Sampling
  4. 2025Generative diffusion posterior sampling for informative likelihoods
  5. 2026Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?
  6. 2026HyperMC: Multi-Fidelity Hyperparameter Tuning for Stochastic Gradient MCMC
  7. 2025Stochastic Weight Sharing for Bayesian Neural Networks
  8. 2025Inverse Problem Sampling in Latent Space Using Sequential Monte Carlo