# Model-Free Local Recalibration of Neural Networks

arXiv: https://arxiv.org/abs/2403.05756 · HTML: https://mlsanity.com/paper/2403.05756/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 9 Mar 2024.

## Builds on (most cited first, 14 of 14)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Classifier Calibration: A survey on how to assess and improve predicted class probabilities](https://mlsanity.com/paper/2112.10327/) (2021)
- [Calibrated and Sharp Uncertainties in Deep Learning via Density Estimation](https://mlsanity.com/paper/2112.07184/) (2021)
- [A Large-Scale Study of Probabilistic Calibration in Neural Network Regression](https://mlsanity.com/paper/2306.02738/) (2023)
- [Proximity-Informed Calibration for Deep Neural Networks](https://mlsanity.com/paper/2306.04590/) (2023)
- [Autoregressive Quantile Flows for Predictive Uncertainty Estimation](https://mlsanity.com/paper/2112.04643/) (2021)
- [Diagnostic tools of approximate Bayesian computation using the coverage property](https://mlsanity.com/paper/1301.3166/) (2013)
- [Quantile Regularization: Towards Implicit Calibration of Regression Models](https://mlsanity.com/paper/2002.12860/) (2020)
- [Recalibration: A post-processing method for approximate Bayesian computation](https://mlsanity.com/paper/1704.06374/) (2017)

## Built on by (most cited first, 1 of 1)

- [Calibrated Multivariate Regression with Localized PIT Mappings](https://mlsanity.com/paper/2409.10855/) (2024)

## Abstract (arXiv)

Artificial neural networks (ANNs) are highly flexible predictive models. However, reliably quantifying uncertainty for their predictions is a continuing challenge. There has been much recent work on "recalibration" of predictive distributions for ANNs, so that forecast probabilities for events of interest are consistent with certain frequency evaluations of them. Uncalibrated probabilistic forecasts are of limited use for many important decision-making tasks. To address this issue, we propose a localized recalibration of ANN predictive distributions using the dimension-reduced representation of the input provided by the ANN hidden layers. Our novel method draws inspiration from recalibration techniques used in the literature on approximate Bayesian computation and likelihood-free inference methods. Most existing calibration methods for ANNs can be thought of as calibrating either on the input layer, which is difficult when the input is high-dimensional, or the output layer, which may not be sufficiently flexible. Through a simulation study, we demonstrate that our method has good performance compared to alternative approaches, and explore the benefits that can be achieved by localizing the calibration based on different layers of the network. Finally, we apply our proposed method to a diamond price prediction problem, demonstrating the potential of our approach to improve prediction and uncertainty quantification in real-world applications.