mlsanityData to 14 Oct 2026

Valid prediction intervals for regression problems

arXiv 2107.00363 ↗First posted 1 Jul 2021stat.ML cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. An important issue is the calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. In this work, we review the above four classes of methods from a conceptual and experimental point of view. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 33 of its 101 references that are in the map. First posted on arXiv 1 Jul 2021.

Builds on

33 of 101 references.

References · 33

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  7. 2016Variational Inference: A Review for Statisticianscited by 1,472
  8. 2019Conformalized Quantile Regressioncited by 629
  9. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 609
  10. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  11. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  12. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
21 more
  1. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  2. 2018Recent Advances in Open Set Recognition: A Surveycited by 310
  3. 2014Scalable Variational Gaussian Process Classificationcited by 228
  4. 2019Predictive inference with the jackknife+cited by 211
  5. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 177
  6. 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 139
  7. 2020Bayesian Neural Networks: An Introduction and Surveycited by 131
  8. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  9. 2013Learning by Transductioncited by 121
  10. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  11. 2019Distribution Calibration for Regressioncited by 89
  12. 2019A comparison of some conformal quantile regression methodscited by 76
  13. 2018Bayesian Uncertainty Estimation for Batch Normalized Deep Networkscited by 70
  14. 2018Exact and Robust Conformal Inference Methods for Predictive Machine Learning With Dependent Datacited by 66
  15. 2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 61
  16. 2014Regression Conformal Prediction with Nearest Neighbourscited by 59
  17. 2019Adaptive, Distribution-Free Prediction Intervals for Deep Networkscited by 56
  18. 2013Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknifecited by 44
  19. 2019Accurate Uncertainty Estimation and Decomposition in Ensemble Learningcited by 42
  20. 2020Quantile Regularization: Towards Implicit Calibration of Regression Modelscited by 8
  21. 2018Calibrated Prediction Intervals for Neural Network Regressorscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 4; 2024: 5; 2025: 10; 2026: 4.

29 papers.

Sort:

  1. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 170
  2. 2024Regression Trees for Fast and Adaptive Prediction Intervalscited by 12
  3. 2022On the Calibration of Probabilistic Classifier Setscited by 8
  4. 2024Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noisecited by 8
  5. 2025Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approachcited by 6
  6. 2022Improved conformalized quantile regressioncited by 5
  7. 2023Conditional validity of heteroskedastic conformal regressioncited by 4
  8. 2022Conformal Prediction Intervals for Remaining Useful Lifetime Estimationcited by 3
  9. 2023Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertaintycited by 3
  10. 2024Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learningcited by 3
  11. 2025Conformal Prediction for Electricity Price Forecasting in the Day-Ahead and Real-Time Balancing Marketcited by 3
  12. 2025Multivariate Latent Recalibration for Conditional Normalizing Flowscited by 3
17 more
  1. 2023Lightweight Regression Model with Prediction Interval Estimation for Computer Vision-based Winter Road Surface Condition Monitoringcited by 2
  2. 2022A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecastingcited by 1
  3. 2025Domain Consistent Industrial Decarbonisation of Global Coal Power Plantscited by 1
  4. 2025Probabilistic Neural Networks (PNNs) with t-Distributed Outputs: Adaptive Prediction Intervals Beyond Gaussian Assumptionscited by 1
  5. 2025A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysiscited by 1
  6. 2025Solar-GECO: Perovskite Solar Cell Property Prediction with Geometric-Aware Co-Attentioncited by 1
  7. 2022Multivariate Prediction Intervals for Random Forestscited by 0
  8. 2023On the Out-of-Distribution Coverage of Combining Split Conformal Prediction and Bayesian Deep Learningcited by 0
  9. 2024Learning from Uncertain Data: From Possible Worlds to Possible Modelscited by 0
  10. 2024Projected random forests and conformal prediction of circular datacited by 0
  11. 2025QUTCC: Quantile Uncertainty Training and Conformal Calibration for Imaging Inverse Problemscited by 0
  12. 2025An Artificial Intelligence Value at Risk Approach: Metrics and Modelscited by 0
  13. 2025The Evolution of Probabilistic Price Forecasting Techniques: A Review of the Day-Ahead, Intra-Day, and Balancing Marketscited by 0
  14. 2026Bi-Level Chaotic Fusion Based Graph Convolutional Network for Stock Market Prediction Intervalcited by 0
  15. 2026Modelling magnetic material properties with uncertainty-aware neural networkscited by 0
  16. 2026Climate-Invariant Conformal Prediction Intervals for Multi-Horizon Solar and Wind Forecastingcited by 0
  17. 2026CALM-5G: Event-Level Conformal Risk Control for Predictive Service-Assurance Alarmscited by 0