mlsanityData to 7 Oct 2026

The Calibration Channel Determines the Bayes-Error Proxy: An Exact Law for Temperature-Induced Distortion

arXiv 2607.18162 ↗First posted 20 Jul 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The soft-label Bayes-error estimator beta(z) = E[min(z, 1-z)] of Ishida et al. estimates the irreducible error of a binary task directly from probability-valued labels. Recent work by Ushio et al. showed that this estimator is fragile when the probabilities are not the true posterior: even perfectly calibrated soft labels can yield a substantially inaccurate estimate, and they propose isotonic calibration as a consistent remedy. We complement that line of work by characterizing exactly how the most widely used post-hoc calibration map -- temperature scaling -- distorts the proxy. We prove an exact, model-free identity reducing the temperature-scaled proxy to the classifier's margin distribution, from which we obtain (i) strict monotonicity in the temperature and (ii) a continuous bijection from the temperature axis onto the open interval (0, 1/2), so that a fixed classifier -- with fixed decisions and fixed 0-1 error -- can be made to report any proxy value whatsoever. Under a Gaussian model of the logits we further derive a two-parameter closed form for the entire proxy-versus-temperature curve. Across CIFAR-10, Fashion-MNIST, and SVHN (eight binary tasks), the proxy varies by 56x to 980x at constant test error, the closed form reproduces the empirical curve to within 0.018, and the calibration temperature that minimizes the expected calibration error does not coincide with any stable proxy value. Our results give a precise, predictive account of the distortion whose existence motivates calibration-based remedies, and they reinforce the practical recommendation that a proxy value is meaningful only together with the mechanism that produced its probabilities.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 13 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

9 of 13 references.

References · 9

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2019Human uncertainty makes classification more robustcited by 231
  6. 2022Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classificationcited by 24
  7. 2014Empirically Estimable Classification Bounds Based on a New Divergence Measurecited by 11
  8. 2018Learning to Bound the Multi-class Bayes Errorcited by 9
  9. 2025Practical estimation of the optimal classification error with soft labels and calibrationcited by 5

Built on by

Not cited yet.