mlsanityData to 7 Oct 2026

Training, Architecture, and Prior for Deterministic Uncertainty Methods

arXiv 2303.05796 ↗First posted 10 Mar 2023cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Accurate and efficient uncertainty estimation is crucial to build reliable Machine Learning (ML) models capable to provide calibrated uncertainty estimates, generalize and detect Out-Of-Distribution (OOD) datasets. To this end, Deterministic Uncertainty Methods (DUMs) is a promising model family capable to perform uncertainty estimation in a single forward pass. This work investigates important design choices in DUMs: (1) we show that training schemes decoupling the core architecture and the uncertainty head schemes can significantly improve uncertainty performances. (2) we demonstrate that the core architecture expressiveness is crucial for uncertainty performance and that additional architecture constraints to avoid feature collapse can deteriorate the trade-off between OOD generalization and detection. (3) Contrary to other Bayesian models, we show that the prior defined by DUMs do not have a strong effect on the final performances.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

39 of 52 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2016Wide Residual Networkscited by 3,057
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  9. 2015Variational Inference with Normalizing Flowscited by 2,161
  10. 2019Invariant Risk Minimizationcited by 1,832
  11. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  12. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
27 more
  1. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  2. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  3. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  4. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  5. 2019Neural Spline Flowscited by 590
  6. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  7. 2015Weight Uncertainty in Neural Networkscited by 554
  8. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  9. 2018Deep Learning for Classical Japanese Literaturecited by 415
  10. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  11. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  12. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  13. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  14. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  15. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
  16. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 163
  17. 2021Bayesian Neural Network Priors Revisitedcited by 121
  18. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  19. 2021Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classificationcited by 79
  20. 2021On the Practicality of Deterministic Epistemic Uncertaintycited by 62
  21. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  22. 2021Resampling Base Distributions of Normalizing Flowscited by 42
  23. 2019Uncertainty on Asynchronous Time Event Predictioncited by 27
  24. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
  25. 2021Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effectcited by 22
  26. 2020A Simple Framework for Uncertainty in Contrastive Learningcited by 16
  27. 2020The Hidden Uncertainty in a Neural Networks Activationscited by 14

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 3.

7 papers.

  1. 2023Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networkscited by 8
  2. 2023Topology-Matching Normalizing Flows for Out-of-Distribution Detection in Robot Learningcited by 7
  3. 2024A Rate-Distortion View of Uncertainty Quantificationcited by 3
  4. 2025Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classificationcited by 3
  5. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
  6. 2024Informed Spectral Normalized Gaussian Processes for Trajectory Predictioncited by 2
  7. 2025Towards Trustworthy Vital Sign Forecasting: Leveraging Uncertainty for Prediction Intervalscited by 2