Inadequacy of common stochastic neural networks for reliable clinical decision support
Not on the map. Why?
Abstract (arXiv)
Widespread adoption of AI for medical decision making is still hindered due to ethical and safety-related concerns. For AI-based decision support systems in healthcare settings it is paramount to be reliable and trustworthy. Common deep learning approaches, however, have the tendency towards overconfidence under data shift. Such inappropriate extrapolation beyond evidence-based scenarios may have dire consequences. This highlights the importance of reliable estimation of local uncertainty and its communication to the end user. While stochastic neural networks have been heralded as a potential solution to these issues, this study investigates their actual reliability in clinical applications. We centered our analysis on the exemplary use case of mortality prediction for ICU hospitalizations using EHR from MIMIC3 study. For predictions on the EHR time series, Encoder-Only Transformer models were employed. Stochasticity of model functions was achieved by incorporating common methods such as Bayesian neural network layers and model ensembles. Our models achieve state of the art performance in terms of discrimination performance (AUC ROC: 0.868+-0.011, AUC PR: 0.554+-0.034) and calibration on the mortality prediction benchmark. However, epistemic uncertainty is critically underestimated by the selected stochastic deep learning methods. A heuristic proof for the responsible collapse of the posterior distribution is provided. Our findings reveal the inadequacy of commonly used stochastic deep learning approaches to reliably recognize OoD samples. In both methods, unsubstantiated model confidence is not prevented due to strongly biased functional posteriors, rendering them inappropriate for reliable clinical decision support. This highlights the need for approaches with more strictly enforced or inherent distance-awareness to known data points, e.g., using kernel-based techniques.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 24 Jan 2024.
Builds on
35 of 54 references.
References · 35
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2016An overview of gradient descent optimization algorithmscited by 1,021
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
- 2015Weight Uncertainty in Neural Networkscited by 554
- 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
- 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
- 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
- 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
- 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
- 2017Multitask learning and benchmarking with clinical time series datacited by 287
23 more
- 2019BEHRT: Transformer for Electronic Health Recordscited by 258
- 2018Explainable Prediction of Medical Codes from Clinical Textcited by 231
- 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 205
- 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
- 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 145
- 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 134
- 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 124
- 2017Attend and Diagnose: Clinical Time Series Analysis using Attention Modelscited by 105
- 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
- 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 75
- 2020Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Reviewcited by 46
- 2019Analyzing the Role of Model Uncertainty for Electronic Health Recordscited by 41
- 2019Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningcited by 37
- 2020Hypermodels for Explorationcited by 37
- 2019TAPER: Time-Aware Patient EHR Representationcited by 22
- 2021On Stein Variational Neural Network Ensemblescited by 21
- 2021Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimationcited by 18
- 2020Deep Bayesian Gaussian Processes for Uncertainty Estimation in Electronic Health Recordscited by 16
- 2019Temporal Self-Attention Network for Medical Concept Embeddingcited by 9
- 2019Inpatient2Vec: Medical Representation Learning for Inpatientscited by 7
- 2020Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networkscited by 7
- 2019Modeling the Uncertainty in Electronic Health Records: a Bayesian Deep Learning Approachcited by 3
- 2022Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networkscited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.