# Valid prediction intervals for regression problems

arXiv: https://arxiv.org/abs/2107.00363 · HTML: https://mlsanity.com/paper/2107.00363/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 101 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 30 of 33)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks](https://mlsanity.com/paper/1511.06314/) (2015)
- [Bayesian Neural Networks: An Introduction and Survey](https://mlsanity.com/paper/2006.12024/) (2020)
- [Analyzing Uncertainty in Neural Machine Translation](https://mlsanity.com/paper/1803.00047/) (2018)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach](https://mlsanity.com/paper/1802.07167/) (2018)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [A comparison of some conformal quantile regression methods](https://mlsanity.com/paper/1909.05433/) (2019)
- [Bayesian Uncertainty Estimation for Batch Normalized Deep Networks](https://mlsanity.com/paper/1802.06455/) (2018)
- [Exact and Robust Conformal Inference Methods for Predictive Machine Learning With Dependent Data](https://mlsanity.com/paper/1802.06300/) (2018)
- [Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control](https://mlsanity.com/paper/1909.09884/) (2019)
- [Regression Conformal Prediction with Nearest Neighbours](https://mlsanity.com/paper/1401.3880/) (2014)
- [Adaptive, Distribution-Free Prediction Intervals for Deep Networks](https://mlsanity.com/paper/1905.10634/) (2019)
- [Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife](https://mlsanity.com/paper/1311.4555/) (2013)

## Built on by (most cited first, 29 of 29)

- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Regression Trees for Fast and Adaptive Prediction Intervals](https://mlsanity.com/paper/2402.07357/) (2024)
- [On the Calibration of Probabilistic Classifier Sets](https://mlsanity.com/paper/2205.10082/) (2022)
- [Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise](https://mlsanity.com/paper/2406.03258/) (2024)
- [Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approach](https://mlsanity.com/paper/2501.06878/) (2025)
- [Improved conformalized quantile regression](https://mlsanity.com/paper/2207.02808/) (2022)
- [Conditional validity of heteroskedastic conformal regression](https://mlsanity.com/paper/2309.08313/) (2023)
- [Conformal Prediction Intervals for Remaining Useful Lifetime Estimation](https://mlsanity.com/paper/2212.14612/) (2022)
- [Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty](https://mlsanity.com/paper/2305.16620/) (2023)
- [Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learning](https://mlsanity.com/paper/2410.03390/) (2024)
- [Conformal Prediction for Electricity Price Forecasting in the Day-Ahead and Real-Time Balancing Market](https://mlsanity.com/paper/2502.04935/) (2025)
- [Multivariate Latent Recalibration for Conditional Normalizing Flows](https://mlsanity.com/paper/2505.16636/) (2025)
- [Lightweight Regression Model with Prediction Interval Estimation for Computer Vision-based Winter Road Surface Condition Monitoring](https://mlsanity.com/paper/2310.00923/) (2023)
- [A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecasting](https://mlsanity.com/paper/2207.14219/) (2022)
- [Domain Consistent Industrial Decarbonisation of Global Coal Power Plants](https://mlsanity.com/paper/2503.03571/) (2025)
- [Probabilistic Neural Networks (PNNs) with t-Distributed Outputs: Adaptive Prediction Intervals Beyond Gaussian Assumptions](https://mlsanity.com/paper/2503.12354/) (2025)
- [A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysis](https://mlsanity.com/paper/2511.00301/) (2025)
- [Solar-GECO: Perovskite Solar Cell Property Prediction with Geometric-Aware Co-Attention](https://mlsanity.com/paper/2511.19263/) (2025)
- [Multivariate Prediction Intervals for Random Forests](https://mlsanity.com/paper/2205.02260/) (2022)
- [On the Out-of-Distribution Coverage of Combining Split Conformal Prediction and Bayesian Deep Learning](https://mlsanity.com/paper/2311.12688/) (2023)
- [Learning from Uncertain Data: From Possible Worlds to Possible Models](https://mlsanity.com/paper/2405.18549/) (2024)
- [Projected random forests and conformal prediction of circular data](https://mlsanity.com/paper/2410.24145/) (2024)
- [QUTCC: Quantile Uncertainty Training and Conformal Calibration for Imaging Inverse Problems](https://mlsanity.com/paper/2507.14760/) (2025)
- [An Artificial Intelligence Value at Risk Approach: Metrics and Models](https://mlsanity.com/paper/2509.18394/) (2025)
- [The Evolution of Probabilistic Price Forecasting Techniques: A Review of the Day-Ahead, Intra-Day, and Balancing Markets](https://mlsanity.com/paper/2511.05523/) (2025)
- [Bi-Level Chaotic Fusion Based Graph Convolutional Network for Stock Market Prediction Interval](https://mlsanity.com/paper/2605.16324/) (2026)
- [Modelling magnetic material properties with uncertainty-aware neural networks](https://mlsanity.com/paper/2606.11870/) (2026)
- [Climate-Invariant Conformal Prediction Intervals for Multi-Horizon Solar and Wind Forecasting](https://mlsanity.com/paper/2607.11470/) (2026)
- [CALM-5G: Event-Level Conformal Risk Control for Predictive Service-Assurance Alarms](https://mlsanity.com/paper/2610.17962/) (2026)

## Abstract (arXiv)

Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. An important issue is the calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. In this work, we review the above four classes of methods from a conceptual and experimental point of view. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.