mlsanityData to 14 Oct 2026

Robustness to corruption in pre-trained Bayesian neural networks

arXiv 2206.12361 ↗First posted 24 Jun 2022cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

We develop ShiftMatch, a new training-data-dependent likelihood for robustness to corruption in Bayesian neural networks (BNNs). ShiftMatch is inspired by the training-data-dependent "EmpCov" priors from Izmailov et al. (2021a), and efficiently matches test-time spatial correlations to those at training time. Critically, ShiftMatch is designed to leave the neural network's training time likelihood unchanged, allowing it to use publicly available samples from pre-trained BNNs. Using pre-trained HMC samples, ShiftMatch gives strong performance improvements on CIFAR-10-C, outperforms EmpCov priors (though ShiftMatch uses extra information from a minibatch of corrupted test points), and is perhaps the first Bayesian method capable of convincingly outperforming plain deep ensembles.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 24 Jun 2022.

Builds on

37 of 57 references.

References · 37

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  7. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  8. 2016End to End Learning for Self-Driving Carscited by 1,009
  9. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  10. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  11. 2019Measuring Calibration in Deep Learningcited by 446
  12. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 431
25 more
  1. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  2. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  3. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  4. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 288
  5. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 236
  6. 2021RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whiteningcited by 215
  7. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  8. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 138
  9. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 138
  10. 2021Repulsive Deep Ensembles are Bayesiancited by 104
  11. 2021The Bayesian Learning Rulecited by 99
  12. 2018Decorrelated Batch Normalizationcited by 83
  13. 2017AutoDIAL: Automatic DomaIn Alignment Layerscited by 75
  14. 2019Unsupervised Domain Adaptation using Feature-Whitening and Consensus Losscited by 67
  15. 2020Revisiting Batch Normalization for Improving Corruption Robustnesscited by 55
  16. 2020A statistical theory of cold posteriors in deep neural networkscited by 54
  17. 2019Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networkscited by 45
  18. 2020Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processescited by 43
  19. 2021Momentum Residual Neural Networkscited by 37
  20. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  21. 2021Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effectcited by 22
  22. 2021Bayesian OOD detection with aleatoric uncertainty and outlier exposurecited by 16
  23. 2021Being a Bit Frequentist Improves Bayesian Neural Networkscited by 15
  24. 2018Bayesian filtering unifies adaptive and non-adaptive neural network optimization methodscited by 9
  25. 2022Tackling covariate shift with node-based Bayesian neural networkscited by 8

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  2. 2024Efficient Test-Time Prompt Tuning for Vision-Language Modelscited by 10
  3. 2023Input-gradient space particle inference for neural network ensemblescited by 7
  4. 2025Are vision language models robust to uncertain inputs?cited by 3
  5. 2024Domain Generalization Guided by Large-Scale Pre-Trained Priorscited by 1