mlsanityData to 7 Oct 2026

Fixing Overconfidence in Dynamic Neural Networks

arXiv 2302.06359 ↗First posted 13 Feb 2023cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the inputs. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100, ImageNet, and Caltech-256 in terms of accuracy, capturing uncertainty, and calibration error.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

38 of 55 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  9. 2016Variational Inference: A Review for Statisticianscited by 1,466
  10. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  11. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
26 more
  1. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  2. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  3. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  4. 2021Dynamic Neural Networks: A Surveycited by 405
  5. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  6. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  7. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  8. 2022Metrics reloaded: Recommendations for image analysis validationcited by 190
  9. 2020Resolution Adaptive Networks for Efficient Inferencecited by 146
  10. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 139
  11. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 125
  12. 2021Bayesian Neural Network Priors Revisitedcited by 121
  13. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  14. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  15. 2020Why should we add early exits to neural networks?cited by 70
  16. 2020The Case for Bayesian Deep Learningcited by 68
  17. 2019Improved Techniques for Training Adaptive Deep Networkscited by 66
  18. 2020A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inferencecited by 61
  19. 2022Uncertainty-guided Source-free Domain Adaptationcited by 46
  20. 2020A Survey of Methods for Low-Power Deep Learning and Computer Visioncited by 37
  21. 2020Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Modelcited by 29
  22. 2022Adapting the Linearised Laplace Model Evidence for Modern Deep Learningcited by 29
  23. 2021Periodic Activation Functions Induce Stationaritycited by 21
  24. 2022Rethinking Bayesian Deep Learning Methods for Semi-Supervised Volumetric Medical Image Segmentationcited by 21
  25. 2021Multi-Exit Semantic Segmentation Networkscited by 18
  26. 2020Stationary Activations for Uncertainty Calibration in Deep Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 7; 2026: 1.

19 papers.

Sort:

By area:

  1. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 27
  2. 2023Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicitycited by 16
  3. 2024Post-hoc Probabilistic Vision-Language Modelscited by 13
  4. 2023Collapsed Inference for Bayesian Deep Learningcited by 6
  5. 2023Early-Exit Neural Networks with Nested Prediction Setscited by 6
  6. 2024Streamlining Prediction in Bayesian Deep Learningcited by 5
  7. 2025A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusioncited by 5
  8. 2024QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoringcited by 4
  9. 2024RAEE: A Robust Retrieval-Augmented Early Exit Framework for Efficient Inferencecited by 4
  10. 2024Dynamic Vocabulary Pruning in Early-Exit LLMscited by 4
  11. 2024Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Onescited by 4
  12. 2025Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliabilitycited by 3
7 more
  1. 2024How to Train Your Multi-Exit Model? Analyzing the Impact of Training Strategiescited by 1
  2. 2025Last Layer Hamiltonian Monte Carlocited by 1
  3. 2025Improving Group Fairness in Knowledge Distillation via Laplace Approximation of Early Exitscited by 0
  4. 2025TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyMLcited by 0
  5. 2025SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyMLcited by 0
  6. 2025Rethinking Calibration for Early-Exit Neural Networkscited by 0
  7. 2026A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networkscited by 0