# Classifier Calibration with ROC-Regularized Isotonic Regression

arXiv: https://arxiv.org/abs/2311.12436 · HTML: https://mlsanity.com/paper/2311.12436/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 37 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Evaluating model calibration in classification](https://mlsanity.com/paper/1902.06977/) (2019)
- [Forecast Hedging and Calibration](https://mlsanity.com/paper/2210.07169/) (2022)
- [Calibrated Forecasts: The Minimax Proof](https://mlsanity.com/paper/2209.05863/) (2022)
- ["Calibeating": Beating Forecasters at Their Own Game](https://mlsanity.com/paper/2209.04892/) (2022)

## Built on by (most cited first, 19 of 19)

- [Rethinking Early Stopping: Refine, Then Calibrate](https://mlsanity.com/paper/2501.19195/) (2025)
- [A Kernel-Based Conditional Two-Sample Test Using Nearest Neighbors (with Applications to Calibration, Regression Curves, and Simulation-Based Inference)](https://mlsanity.com/paper/2407.16550/) (2024)
- [Distributionally robust risk evaluation with an isotonic constraint](https://mlsanity.com/paper/2407.06867/) (2024)
- [Efficient Fairness-Performance Pareto Front Computation](https://mlsanity.com/paper/2409.17643/) (2024)
- [Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement](https://mlsanity.com/paper/2505.01766/) (2025)
- [Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling](https://mlsanity.com/paper/2502.15131/) (2025)
- [A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage](https://mlsanity.com/paper/2512.19142/) (2025)
- [CalArena: A Large-Scale Post-Hoc Calibration Benchmark](https://mlsanity.com/paper/2605.30188/) (2026)
- [Predicting the Lifespan of Industrial Printheads with Survival Analysis](https://mlsanity.com/paper/2504.07638/) (2025)
- [Non-collective Calibrating Strategy for Time Series Forecasting](https://mlsanity.com/paper/2506.03176/) (2025)
- [A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysis](https://mlsanity.com/paper/2511.00301/) (2025)
- [Brenier Isotonic Regression](https://mlsanity.com/paper/2603.10452/) (2026)
- [Forecasting West Nile virus with deep graph encoders](https://mlsanity.com/paper/2509.22657/) (2025)
- [CRACQ: A Multi-Dimensional Approach To Automated Document Assessment](https://mlsanity.com/paper/2510.02337/) (2025)
- [Causal-Audit: A Framework for Risk Assessment of Assumption Violations in Time-Series Causal Discovery](https://mlsanity.com/paper/2604.02488/) (2026)
- [An Empirical Study of Security Calibration in Large Language Models for Code](https://mlsanity.com/paper/2606.31159/) (2026)
- [TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings](https://mlsanity.com/paper/2607.11007/) (2026)
- [An analysis of binary isotonic regression: degrees of freedom and implications for calibration](https://mlsanity.com/paper/2607.27301/) (2026)
- [PICPIs: Prediction-Interval-Conditional Prediction Intervals](https://mlsanity.com/paper/2609.25388/) (2026)

## Abstract (arXiv)

Calibration of machine learning classifiers is necessary to obtain reliable and interpretable predictions, bridging the gap between model confidence and actual probabilities. One prominent technique, isotonic regression (IR), aims at calibrating binary classifiers by minimizing the cross entropy on a calibration set via monotone transformations. IR acts as an adaptive binning procedure, which allows achieving a calibration error of zero, but leaves open the issue of the effect on performance. In this paper, we first prove that IR preserves the convex hull of the ROC curve -- an essential performance metric for binary classifiers. This ensures that a classifier is calibrated while controlling for overfitting of the calibration set. We then present a novel generalization of isotonic regression to accommodate classifiers with K classes. Our method constructs a multidimensional adaptive binning scheme on the probability simplex, again achieving a multi-class calibration error equal to zero. We regularize this algorithm by imposing a form of monotony that preserves the K-dimensional ROC surface of the classifier. We show empirically that this general monotony criterion is effective in striking a balance between reducing cross entropy loss and avoiding overfitting of the calibration set.