Be Bayesian by Attachments to Catch More Uncertainty
Not on the map. Why?
Abstract (arXiv)
Bayesian Neural Networks (BNNs) have become one of the promising approaches for uncertainty estimation due to the solid theorical foundations. However, the performance of BNNs is affected by the ability of catching uncertainty. Instead of only seeking the distribution of neural network weights by in-distribution (ID) data, in this paper, we propose a new Bayesian Neural Network with an Attached structure (ABNN) to catch more uncertainty from out-of-distribution (OOD) data. We first construct a mathematical description for the uncertainty of OOD data according to the prior distribution, and then develop an attached Bayesian structure to integrate the uncertainty of OOD data into the backbone network. ABNN is composed of an expectation module and several distribution modules. The expectation module is a backbone deep network which focuses on the original task, and the distribution modules are mini Bayesian structures which serve as attachments of the backbone. In particular, the distribution modules aim at extracting the uncertainty from both ID and OOD data. We further provide theoretical analysis for the convergence of ABNN, and experimentally validate its superiority by comparing with some state-of-the-art uncertainty estimation methods Code will be made available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 96 references that are in the map. First posted on arXiv 19 Oct 2023.
Builds on
57 of 96 references.
References · 57
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
- 2020Energy-based Out-of-distribution Detectioncited by 1,299
- 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
- 2018Deep Anomaly Detection with Outlier Exposurecited by 942
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
- 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
- 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
38 more
- 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
- 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
- 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
- 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
- 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
- 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 252
- 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 231
- 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
- 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
- 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
- 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
- 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
- 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
- 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
- 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
- 2016Deep Label Distribution Learning with Label Ambiguitycited by 104
- 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
- 2021Cluster Contrast for Unsupervised Person Re-Identificationcited by 89
- 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
- 2022Training OOD Detectors in their Natural Habitatscited by 86
- 2020SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimatescited by 75
- 2021Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environmentscited by 66
- 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
- 2020What Do Neural Networks Learn When Trained With Random Labels?cited by 59
- 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 53
- 2019A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inferencecited by 51
- 2022Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classificationcited by 50
- 2022Latent Outlier Exposure for Anomaly Detection with Contaminated Datacited by 48
- 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
- 2018Rethinking Feature Distribution for Loss Functions in Image Classificationcited by 39
- 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
- 2020Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descentcited by 35
- 2019Specifying Weight Priors in Bayesian Deep Neural Networks with Empirical Bayescited by 32
- 2022Uncertainty-aware Label Distribution Learning for Facial Expression Recognitioncited by 30
- 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 28
- 2021A Bit More Bayesian: Domain-Invariant Learning with Uncertaintycited by 27
- 2019Novelty Detection Via Blurringcited by 24
- 2022Out-of-distribution Detection via Frequency-regularized Generative Modelscited by 23
50 most cited of 57 All 57 (JSON)
Built on by
Not cited yet.