mlsanityData to 14 Oct 2026

Probabilistic Calibration by Design for Neural Network Regression

arXiv 2403.11964 ↗First posted 18 Mar 2024cs.LG stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Generating calibrated and sharp neural network predictive distributions for regression problems is essential for optimal decision-making in many real-world applications. To address the miscalibration issue of neural networks, various methods have been proposed to improve calibration, including post-hoc methods that adjust predictions after training and regularization methods that act during training. While post-hoc methods have shown better improvement in calibration compared to regularization methods, the post-hoc step is completely independent of model training. We introduce a novel end-to-end model training procedure called Quantile Recalibration Training, integrating post-hoc calibration directly into the training process without additional parameters. We also present a unified algorithm that includes our method and other post-hoc and regularization methods, as particular cases. We demonstrate the performance of our method in a large-scale experiment involving 57 tabular regression datasets, showcasing improved predictive accuracy while maintaining calibration. We also conduct an ablation study to evaluate the significance of different components within our proposed method, as well as an in-depth analysis of the impact of the base model and different hyperparameters on predictive accuracy.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 32 of its 57 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

32 of 57 references.

References · 32

  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  5. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  6. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  7. 2019Conformalized Quantile Regressioncited by 629
  8. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
  9. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  10. 2019Deep Evidential Regressioncited by 355
  11. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 199
  12. 2018Neural Importance Samplingcited by 155
20 more
  1. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  2. 2015Should we really use post-hoc tests based on mean-ranks?cited by 116
  3. 2021Learning Optimal Conformal Classifierscited by 112
  4. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  5. 2019Distributional conformal predictioncited by 105
  6. 2020Calibration of Neural Networks using Splinescited by 97
  7. 2021Soft Calibration Objectives for Neural Networkscited by 92
  8. 2019An Open Source AutoML Benchmarkcited by 85
  9. 2020Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantificationcited by 78
  10. 2019Flexible distribution-free conditional predictive bands using density estimatorscited by 64
  11. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 64
  12. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 63
  13. 2018Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Controlcited by 55
  14. 2020Individual Calibration with Randomized Forecastingcited by 50
  15. 2022A Consistent and Differentiable Lp Canonical Calibration Error Estimatorcited by 47
  16. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  17. 2023A Large-Scale Study of Probabilistic Calibration in Neural Network Regressioncited by 27
  18. 2023ESD: Expected Squared Difference as a Tuning-Free Trainable Calibration Measurecited by 13
  19. 2020Beyond Strictly Proper Scoring Rules: The Importance of Being Localcited by 9
  20. 2020Quantile Regularization: Towards Implicit Calibration of Regression Modelscited by 8

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2025Rectifying Conformity Scores for Better Conditional Coveragecited by 17
  2. 2024Conformal Risk Minimization with Variance Reductioncited by 11
  3. 2025Multivariate Latent Recalibration for Conditional Normalizing Flowscited by 3
  4. 2024A Decision-driven Methodology for Designing Uncertainty-aware AI Self-Assessmentcited by 1
  5. 2025Bayesian Inference of Primordial Magnetic Field Parameters from CMB with Spherical Graph Neural Networkscited by 0
  6. 2025Enforcing Calibration in Multi-Output Probabilistic Regression with Pre-rank Regularizationcited by 0