# Using Platt's scaling for calibration after undersampling -- limitations and how to address them

arXiv: https://arxiv.org/abs/2410.18144 · HTML: https://mlsanity.com/paper/2410.18144/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 36 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 4 of 4)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Spline-Based Probability Calibration](https://mlsanity.com/paper/1809.07751/) (2018)
- [Probability Calibration Trees](https://mlsanity.com/paper/1808.00111/) (2018)

## Built on by (most cited first, 5 of 5)

- [HFMF: Hierarchical Fusion Meets Multi-Stream Models for Deepfake Detection](https://mlsanity.com/paper/2501.05631/) (2025)
- [Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling](https://mlsanity.com/paper/2502.15131/) (2025)
- [An Oversampling-enhanced Multi-class Imbalanced Classification Framework for Patient Health Status Prediction Using Patient-reported Outcomes](https://mlsanity.com/paper/2411.10819/) (2024)
- [Analyzing decision tree bias towards the minority class](https://mlsanity.com/paper/2501.04903/) (2025)
- [Machine Learning Classification of Cryopathy Syndromes: A Comprehensive Comparative Study](https://mlsanity.com/paper/2606.20874/) (2026)

## Abstract (arXiv)

When modelling data where the response is dichotomous and highly imbalanced, response-based sampling where a subset of the majority class is retained (i.e., undersampling) is often used to create more balanced training datasets prior to modelling. However, the models fit to this undersampled data, which we refer to as base models, generate predictions that are severely biased. There are several calibration methods that can be used to combat this bias, one of which is Platt's scaling. Here, a logistic regression model is used to model the relationship between the base model's original predictions and the response. Despite its popularity for calibrating models after undersampling, Platt's scaling was not designed for this purpose. Our work presents what we believe is the first detailed study focused on the validity of using Platt's scaling to calibrate models after undersampling. We show analytically, as well as via a simulation study and a case study, that Platt's scaling should not be used for calibration after undersampling without critical thought. If Platt's scaling would have been able to successfully calibrate the base model had it been trained on the entire dataset (i.e., without undersampling), then Platt's scaling might be appropriate for calibration after undersampling. If this is not the case, we recommend a modified version of Platt's scaling that fits a logistic generalized additive model to the logit of the base model's predictions, as it is both theoretically motivated and performed well across the settings considered in our study.