# Can bin-wise scaling improve consistency and adaptivity of prediction uncertainty for machine learning regression ?

arXiv: https://arxiv.org/abs/2310.11978 · HTML: https://mlsanity.com/paper/2310.11978/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Evaluating and Calibrating Uncertainty Prediction in Regression Tasks](https://mlsanity.com/paper/1905.11659/) (2019)
- [Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification](https://mlsanity.com/paper/2109.10254/) (2021)
- [Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networks](https://mlsanity.com/paper/2107.06068/) (2021)
- [Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity](https://mlsanity.com/paper/2309.06240/) (2023)
- [Properties of the ENCE and other MAD-based calibration metrics](https://mlsanity.com/paper/2305.11905/) (2023)
- [Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statistics](https://mlsanity.com/paper/2306.05180/) (2023)
- [Validation of uncertainty quantification metrics: a primer based on the consistency and adaptivity concepts](https://mlsanity.com/paper/2303.07170/) (2023)
- [The Shannon Entropy of a Histogram](https://mlsanity.com/paper/2210.02848/) (2022)
- [ELUQuant: Event-Level Uncertainty Quantification in Deep Inelastic Scattering](https://mlsanity.com/paper/2310.02913/) (2023)

## Built on by (most cited first, 4 of 4)

- [Negative impact of heavy-tailed uncertainty and error distributions on the reliability of calibration statistics for machine learning regression tasks](https://mlsanity.com/paper/2402.10043/) (2024)
- [Validation of ML-UQ calibration statistics using simulated reference values: a sensitivity analysis](https://mlsanity.com/paper/2403.00423/) (2024)
- [Robust Taxi Fare Prediction Under Noisy Conditions: A Comparative Study of GAT, TimesNet, and XGBoost](https://mlsanity.com/paper/2507.20008/) (2025)
- [Trustworthy deep domain adaptation for wearable photoplethysmography signal analysis with decision-theoretic uncertainty quantification](https://mlsanity.com/paper/2604.17480/) (2026)

## Abstract (arXiv)

Binwise Variance Scaling (BVS) has recently been proposed as a post hoc recalibration method for prediction uncertainties of machine learning regression problems that is able of more efficient corrections than uniform variance (or temperature) scaling. The original version of BVS uses uncertainty-based binning, which is aimed to improve calibration conditionally on uncertainty, i.e. consistency. I explore here several adaptations of BVS, in particular with alternative loss functions and a binning scheme based on an input-feature (X) in order to improve adaptivity, i.e. calibration conditional on X. The performances of BVS and its proposed variants are tested on a benchmark dataset for the prediction of atomization energies and compared to the results of isotonic regression.