# PI3NN: Out-of-distribution-aware prediction intervals from three neural networks

arXiv: https://arxiv.org/abs/2108.02327 · HTML: https://mlsanity.com/paper/2108.02327/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 11 of 11)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach](https://mlsanity.com/paper/1802.07167/) (2018)
- [Uncertainty in Neural Networks: Approximately Bayesian Ensembling](https://mlsanity.com/paper/1810.05546/) (2018)
- [Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles](https://mlsanity.com/paper/2007.09670/) (2020)
- [PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction](https://mlsanity.com/paper/2006.05139/) (2020)

## Built on by (most cited first, 5 of 5)

- [Anomaly Detection with Score Distribution Discrimination](https://mlsanity.com/paper/2306.14403/) (2023)
- [Adaptive Sampling to Reduce Epistemic Uncertainty Using Prediction Interval-Generation Neural Networks](https://mlsanity.com/paper/2412.10570/) (2024)
- [Scalable Artificial Intelligence for Science: Perspectives, Methods and Exemplars](https://mlsanity.com/paper/2406.17812/) (2024)
- [SEF: A Method for Computing Prediction Intervals by Shifting the Error Function in Neural Networks](https://mlsanity.com/paper/2409.05206/) (2024)
- [Barrier-enforced multi-objective optimization for direct point and sharp interval forecasting](https://mlsanity.com/paper/2604.18492/) (2026)

## Abstract (arXiv)

We propose a novel prediction interval (PI) method for uncertainty quantification, which addresses three major issues with the state-of-the-art PI methods. First, existing PI methods require retraining of neural networks (NNs) for every given confidence level and suffer from the crossing issue in calculating multiple PIs. Second, they usually rely on customized loss functions with extra sensitive hyperparameters for which fine tuning is required to achieve a well-calibrated PI. Third, they usually underestimate uncertainties of out-of-distribution (OOD) samples leading to over-confident PIs. Our PI3NN method calculates PIs from linear combinations of three NNs, each of which is independently trained using the standard mean squared error loss. The coefficients of the linear combinations are computed using root-finding algorithms to ensure tight PIs for a given confidence level. We theoretically prove that PI3NN can calculate PIs for a series of confidence levels without retraining NNs and it completely avoids the crossing issue. Additionally, PI3NN does not introduce any unusual hyperparameters resulting in a stable performance. Furthermore, we address OOD identification challenge by introducing an initialization scheme which provides reasonably larger PIs of the OOD samples than those of the in-distribution samples. Benchmark and real-world experiments show that our method outperforms several state-of-the-art approaches with respect to predictive uncertainty quality, robustness, and OOD samples identification.