mlsanityData to 7 Oct 2026

Conformal inference is (almost) free for neural networks trained with early stopping

arXiv 2301.11556 ↗First posted 27 Jan 2023stat.ML cs.LG math.STcited by 14

Not on the map. Why?

Abstract (arXiv)

Early stopping based on hold-out data is a popular regularization technique designed to mitigate overfitting and increase the predictive accuracy of neural networks. Models trained with early stopping often provide relatively accurate predictions, but they generally still lack precise statistical guarantees unless they are further calibrated using independent hold-out data. This paper addresses the above limitation with conformalized early stopping: a novel method that combines early stopping with conformal calibration while efficiently recycling the same hold-out data. This leads to models that are both accurate and able to provide exact predictive inferences without multiple data splits nor overly conservative adjustments. Practical implementations are developed for different learning tasks -- outlier detection, multi-class classification, regression -- and their competitive performance is demonstrated on real data.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 32 of its 73 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

32 of 73 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  5. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  6. 2019Conformalized Quantile Regressioncited by 620
  7. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  8. 2019Conformal Prediction Under Covariate Shiftcited by 511
  9. 2020Classification with Valid and Adaptive Coveragecited by 390
  10. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  11. 2022Conformal prediction beyond exchangeabilitycited by 348
  12. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
20 more
  1. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  2. 2018Deep Learning: A Critical Appraisalcited by 214
  3. 2019Predictive inference with the jackknife+cited by 206
  4. 2021Testing for Outliers with Conformal p-valuescited by 133
  5. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 123
  6. 2021Learning Optimal Conformal Classifierscited by 108
  7. 2019Distributional conformal predictioncited by 104
  8. 2021Conformal Prediction using Conditional Histogramscited by 77
  9. 2019A comparison of some conformal quantile regression methodscited by 76
  10. 2019Flexible distribution-free conditional predictive bands using density estimatorscited by 64
  11. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 62
  12. 2020Predictive Inference Is Free with the Jackknife+-after-Bootstrapcited by 55
  13. 2021Selection and Aggregation of Conformal Prediction Setscited by 46
  14. 2019Prediction and outlier detection in classification problemscited by 44
  15. 2019With Malice Towards None: Assessing Uncertainty via Equalized Coveragecited by 43
  16. 2018Cautious Deep Learningcited by 35
  17. 2021Optimized conformal classification using gradient descent approximationcited by 27
  18. 2020Training conformal predictorscited by 20
  19. 2022Integrative conformal p-values for powerful out-of-distribution testing with labeled outlierscited by 19
  20. 2022Conformal Frequency Estimation with Sketched Datacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 6; 2026: 4.

14 papers.

Sort:

By area:

  1. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  2. 2024Conformal Prediction: A Data Perspectivecited by 25
  3. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 11
  4. 2025Synthetic-Powered Predictive Inferencecited by 9
  5. 2023SymmPI: Predictive Inference for Data with Group Symmetriescited by 8
  6. 2025Parametric Scaling Law of Tuning Bias in Conformal Predictioncited by 7
  7. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  8. 2025Singleton-Optimized Conformal Predictioncited by 2
  9. 2025Interpretable Multivariate Conformal Prediction with Fast Transductive Standardizationcited by 2
  10. 2024Energy Estimation of Last Mile Electric Vehicle Routescited by 1
  11. 2026Softmax is not Enough (for Adaptive Conformal Classification)cited by 0
  12. 2026Elements of Conformal Predictioncited by 0
2 more
  1. 2026Though Language Models Err While They Strive: Conformal Prediction for Self-Correcting Scientific Generationcited by 0
  2. 2026Where A Small Language Model Helps in Invoice Categorisation, Understood Through Embedding Geometrycited by 0