# Distribution Calibration for Regression

arXiv: https://arxiv.org/abs/1905.06023 · HTML: https://mlsanity.com/paper/1905.06023/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 30 of 89)

- [Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration](https://mlsanity.com/paper/2107.05719/) (2021)
- [Classifier Calibration: A survey on how to assess and improve predicted class probabilities](https://mlsanity.com/paper/2112.10327/) (2021)
- [Better Uncertainty Calibration via Proper Scores for Classification and Beyond](https://mlsanity.com/paper/2203.07835/) (2022)
- [Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification](https://mlsanity.com/paper/2109.10254/) (2021)
- [Low-Degree Multicalibration](https://mlsanity.com/paper/2203.01255/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Calibrated and Sharp Uncertainties in Deep Learning via Density Estimation](https://mlsanity.com/paper/2112.07184/) (2021)
- [X-CAL: Explicit Calibration for Survival Analysis](https://mlsanity.com/paper/2101.05346/) (2021)
- [Valid prediction intervals for regression problems](https://mlsanity.com/paper/2107.00363/) (2021)
- [A Large-Scale Study of Probabilistic Calibration in Neural Network Regression](https://mlsanity.com/paper/2306.02738/) (2023)
- [Estimating Expected Calibration Errors](https://mlsanity.com/paper/2109.03480/) (2021)
- [Flexible Model Aggregation for Quantile Regression](https://mlsanity.com/paper/2103.00083/) (2021)
- [COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition](https://mlsanity.com/paper/2206.05833/) (2022)
- [Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networks](https://mlsanity.com/paper/2107.06068/) (2021)
- [Calibration tests beyond classification](https://mlsanity.com/paper/2210.13355/) (2022)
- [Calibration by Distribution Matching: Trainable Kernel Calibration Metrics](https://mlsanity.com/paper/2310.20211/) (2023)
- [Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration](https://mlsanity.com/paper/2307.00934/) (2023)
- [CAMul: Calibrated and Accurate Multi-view Time-Series Forecasting](https://mlsanity.com/paper/2109.07438/) (2021)
- [When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting](https://mlsanity.com/paper/2106.03904/) (2021)
- [Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions](https://mlsanity.com/paper/2105.04471/) (2021)
- [A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting](https://mlsanity.com/paper/2206.08998/) (2022)
- [Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts](https://mlsanity.com/paper/2302.06495/) (2023)
- [Guaranteed Coverage Prediction Intervals with Gaussian Process Regression](https://mlsanity.com/paper/2310.15641/) (2023)
- [Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis](https://mlsanity.com/paper/2311.01052/) (2023)
- [MUBen: Benchmarking the Uncertainty of Molecular Representation Models](https://mlsanity.com/paper/2306.10060/) (2023)
- [Uncertainty Estimation for Molecules: Desiderata and Methods](https://mlsanity.com/paper/2306.14916/) (2023)
- [Online Calibrated and Conformal Prediction Improves Bayesian Optimization](https://mlsanity.com/paper/2112.04620/) (2021)
- [Evaluating Aleatoric Uncertainty via Conditional Generative Models](https://mlsanity.com/paper/2206.04287/) (2022)
- [On the Calibration of Human Pose Estimation](https://mlsanity.com/paper/2311.17105/) (2023)
- [Parametric and Multivariate Uncertainty Calibration for Regression and Object Detection](https://mlsanity.com/paper/2207.01242/) (2022)

## Abstract (arXiv)

We are concerned with obtaining well-calibrated output distributions from regression models. Such distributions allow us to quantify the uncertainty that the model has regarding the predicted target value. We introduce the novel concept of distribution calibration, and demonstrate its advantages over the existing definition of quantile calibration. We further propose a post-hoc approach to improving the predictions from previously trained regression models, using multi-output Gaussian Processes with a novel Beta link function. The proposed method is experimentally verified on a set of common regression models and shows improvements for both distribution-level and quantile-level calibration.