# Binary Classifier Calibration: Non-parametric approach

arXiv: https://arxiv.org/abs/1401.3390 · HTML: https://mlsanity.com/paper/1401.3390/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2014.

## Built on by (most cited first, 13 of 13)

- [Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing](https://mlsanity.com/paper/2309.12236/) (2023)
- [A Unifying Theory of Distance from Calibration](https://mlsanity.com/paper/2211.16886/) (2022)
- [Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation](https://mlsanity.com/paper/2311.08877/) (2023)
- [Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift](https://mlsanity.com/paper/2207.08977/) (2022)
- [Analysis and Comparison of Classification Metrics](https://mlsanity.com/paper/2209.05355/) (2022)
- [MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools](https://mlsanity.com/paper/2504.20168/) (2025)
- [Evaluating Posterior Probabilities: Decision Theory, Proper Scoring Rules, and Calibration](https://mlsanity.com/paper/2408.02841/) (2024)
- [MBCT: Tree-Based Feature-Aware Binning for Individual Uncertainty Calibration](https://mlsanity.com/paper/2202.04348/) (2022)
- [Reassessing How to Compare and Improve the Calibration of Machine Learning Models](https://mlsanity.com/paper/2406.04068/) (2024)
- [Reducing Overconfidence Predictions for Autonomous Driving Perception](https://mlsanity.com/paper/2202.07825/) (2022)
- [How Flawed Is ECE? An Analysis via Logit Smoothing](https://mlsanity.com/paper/2402.10046/) (2024)
- [GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networks](https://mlsanity.com/paper/2410.09570/) (2024)
- [Non-Parametric Machine Text Detection via Multi-View Gaussian Processes](https://mlsanity.com/paper/2606.14060/) (2026)

## Abstract (arXiv)

Accurate calibration of probabilistic predictive models learned is critical for many practical prediction and decision-making tasks. There are two main categories of methods for building calibrated classifiers. One approach is to develop methods for learning probabilistic models that are well-calibrated, ab initio. The other approach is to use some post-processing methods for transforming the output of a classifier to be well calibrated, as for example histogram binning, Platt scaling, and isotonic regression. One advantage of the post-processing approach is that it can be applied to any existing probabilistic classification model that was constructed using any machine-learning method. In this paper, we first introduce two measures for evaluating how well a classifier is calibrated. We prove three theorems showing that using a simple histogram binning post-processing method, it is possible to make a classifier be well calibrated while retaining its discrimination capability. Also, by casting the histogram binning method as a density-based non-parametric binary classifier, we can extend it using two simple non-parametric density estimation methods. We demonstrate the performance of the proposed calibration methods on synthetic and real datasets. Experimental results show that the proposed methods either outperform or are comparable to existing calibration methods.