# The harms of class imbalance corrections for machine learning based prediction models: a simulation study

arXiv: https://arxiv.org/abs/2404.19494 · HTML: https://mlsanity.com/paper/2404.19494/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2024.

## Built on by (most cited first, 10 of 10)

- [Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models](https://mlsanity.com/paper/2510.03136/) (2025)
- [MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities](https://mlsanity.com/paper/2505.12043/) (2025)
- [Electrocardiogram-based diagnosis of liver diseases: an externally validated and explainable machine learning approach](https://mlsanity.com/paper/2412.03717/) (2024)
- [Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models](https://mlsanity.com/paper/2510.11222/) (2025)
- [Explainable bank failure prediction models: Counterfactual explanations to reduce the failure risk](https://mlsanity.com/paper/2407.11089/) (2024)
- [On Inverse Problems, Parameter Estimation, and Domain Generalization](https://mlsanity.com/paper/2506.06024/) (2025)
- [A Theoretical and Empirical Taxonomy of Imbalance in Binary Classification](https://mlsanity.com/paper/2601.04149/) (2026)
- [Tipping the Balance: Impact of Class Imbalance Correction on the Performance of Clinical Risk Prediction Models](https://mlsanity.com/paper/2603.00208/) (2026)
- [Digitally enriching a high-risk population for pancreatic cancer using routine blood-based measures and clinical histories](https://mlsanity.com/paper/2605.30275/) (2026)
- [PICPIs: Prediction-Interval-Conditional Prediction Intervals](https://mlsanity.com/paper/2609.25388/) (2026)

## Abstract (arXiv)

Risk prediction models are increasingly used in healthcare to aid in clinical decision making. In most clinical contexts, model calibration (i.e., assessing the reliability of risk estimates) is critical. Data available for model development are often not perfectly balanced with respect to the modeled outcome (i.e., individuals with vs. without the event of interest are not equally represented in the data). It is common for researchers to correct this class imbalance, yet, the effect of such imbalance corrections on the calibration of machine learning models is largely unknown. We studied the effect of imbalance corrections on model calibration for a variety of machine learning algorithms. Using extensive Monte Carlo simulations we compared the out-of-sample predictive performance of models developed with an imbalance correction to those developed without a correction for class imbalance across different data-generating scenarios (varying sample size, the number of predictors and event fraction). Our findings were illustrated in a case study using MIMIC-III data. In all simulation scenarios, prediction models developed without a correction for class imbalance consistently had equal or better calibration performance than prediction models developed with a correction for class imbalance. The miscalibration introduced by correcting for class imbalance was characterized by an over-estimation of risk and was not always able to be corrected with re-calibration. Correcting for class imbalance is not always necessary and may even be harmful for clinical prediction models which aim to produce reliable risk estimates on an individual basis.