mlsanityData to 7 Oct 2026

Calibrating a Deep Neural Network with Its Predecessors

arXiv 2302.06245 ↗First posted 13 Feb 2023cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Confidence calibration - the process to calibrate the output probability distribution of neural networks - is essential for safety-critical applications of such networks. Recent works verify the link between mis-calibration and overfitting. However, early stopping, as a well-known technique to mitigate overfitting, fails to calibrate networks. In this work, we study the limitions of early stopping and comprehensively analyze the overfitting problem of a network considering each individual block. We then propose a novel regularization method, predecessor combination search (PCS), to improve calibration by searching a combination of best-fitting block predecessors, where block predecessors are the corresponding network blocks with weight parameters from earlier training stages. PCS achieves the state-of-the-art calibration performance on multiple datasets and architectures. In addition, PCS improves model robustness under dataset distribution shift.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

37 of 62 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016Wide Residual Networkscited by 3,057
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  9. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  12. 2016End to End Learning for Self-Driving Carscited by 1,000
25 more
  1. 2019When Does Label Smoothing Help?cited by 985
  2. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  3. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  4. 2019Measuring Calibration in Deep Learningcited by 443
  5. 2020Meta Pseudo Labelscited by 372
  6. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  7. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  8. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  9. 2019Verified Uncertainty Calibrationcited by 307
  10. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  11. 2020DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolutioncited by 222
  12. 2020Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Taskscited by 137
  13. 2021Probabilistic two-stage detectioncited by 130
  14. 2020Improving model calibration with accuracy versus uncertainty optimizationcited by 123
  15. 2020Mitigating Bias in Calibration Error Estimationcited by 100
  16. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  17. 2020Calibration of Neural Networks using Splinescited by 97
  18. 2021Soft Calibration Objectives for Neural Networkscited by 89
  19. 2021You Only Learn One Representation: Unified Network for Multiple Taskscited by 86
  20. 2013Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networkscited by 84
  21. 2019Non-Parametric Calibration for Classificationcited by 57
  22. 2020Distilling portable Generative Adversarial Networks for Image Translationcited by 36
  23. 2020A Semi-Supervised Assessor of Neural Architecturescited by 28
  24. 2020Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNscited by 9
  25. 2020Learning with Retrospectioncited by 9

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 5.

9 papers.

  1. 2023Dual Focal Loss for Calibrationcited by 33
  2. 2023A Benchmark Study on Calibrationcited by 10
  3. 2025Uncertainty Weighted Gradients for Model Calibrationcited by 5
  4. 2024Consistency Calibration: Improving Uncertainty Calibration via Consistency among Perturbed Neighborscited by 3
  5. 2025Balancing Two Classifiers via A Simplex ETF Structure for Model Calibrationcited by 3
  6. 2025h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objectivecited by 2
  7. 2025Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Maskingcited by 1
  8. 2025Target-Agnostic Calibration under Distribution Shift with Frequency-Aware Gradient Rectificationcited by 1
  9. 2024Parametric ρ-Norm Scaling Calibrationcited by 0