# Is this model reliable for everyone? Testing for strong calibration

arXiv: https://arxiv.org/abs/2307.15247 · HTML: https://mlsanity.com/paper/2307.15247/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 81 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Builds on (most cited first, 27 of 27)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Multiaccuracy: Black-Box Post-Processing for Fairness in Classification](https://mlsanity.com/paper/1805.12317/) (2018)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction](https://mlsanity.com/paper/2106.08460/) (2021)
- [Calibration tests in multi-class classification: A unifying framework](https://mlsanity.com/paper/1910.11385/) (2019)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [Learning Certified Individually Fair Representations](https://mlsanity.com/paper/2002.10312/) (2020)
- [Efficient nonparametric statistical inference on population feature importance using Shapley values](https://mlsanity.com/paper/2006.09481/) (2020)
- [Distributionally Robust Losses for Latent Covariate Mixtures](https://mlsanity.com/paper/2007.13982/) (2020)
- [Metric Learning for Individual Fairness](https://mlsanity.com/paper/1906.00250/) (2019)
- [Individual Calibration with Randomized Forecasting](https://mlsanity.com/paper/2006.10288/) (2020)
- [With Malice Towards None: Assessing Uncertainty via Equalized Coverage](https://mlsanity.com/paper/1908.05428/) (2019)
- [Multicalibration as Boosting for Regression](https://mlsanity.com/paper/2301.13767/) (2023)
- [Local Calibration: Metrics and Recalibration](https://mlsanity.com/paper/2102.10809/) (2021)
- [T-Cal: An optimal test for the calibration of predictive models](https://mlsanity.com/paper/2203.01850/) (2022)
- [Conformal calibrators](https://mlsanity.com/paper/1902.06579/) (2019)
- [Auditing ML Models for Individual Bias and Unfairness](https://mlsanity.com/paper/2003.05048/) (2020)
- [Modular Conformal Calibration](https://mlsanity.com/paper/2206.11468/) (2022)
- [Statistical inference for individual fairness](https://mlsanity.com/paper/2103.16714/) (2021)
- [Is a Classification Procedure Good Enough? A Goodness-of-Fit Assessment Tool for Classification Learning](https://mlsanity.com/paper/1911.03063/) (2019)
- [Goodness-of-fit testing in high-dimensional generalized linear models](https://mlsanity.com/paper/1908.03606/) (2019)
- [Inference on function-valued parameters using a restricted score test](https://mlsanity.com/paper/2105.06646/) (2021)
- [Monitoring machine learning (ML)-based risk prediction algorithms in the presence of confounding medical interventions](https://mlsanity.com/paper/2211.09781/) (2022)

## Built on by (most cited first, 6 of 6)

- [Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens](https://mlsanity.com/paper/2311.11463/) (2023)
- [Adaptive auditing of AI systems with anytime-valid guarantees](https://mlsanity.com/paper/2605.07002/) (2026)
- [Algorithmic Accountability in Small Data: Sample-Size-Induced Bias Within Classification Metrics](https://mlsanity.com/paper/2505.03992/) (2025)
- ["Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift](https://mlsanity.com/paper/2506.00756/) (2025)
- [On the Role of Calibration in Benchmarking Algorithmic Fairness for Skin Cancer Detection](https://mlsanity.com/paper/2511.07700/) (2025)
- [Reliably Detecting Model Failures in Deployment Without Labels](https://mlsanity.com/paper/2506.05047/) (2025)

## Abstract (arXiv)

In a well-calibrated risk prediction model, the average predicted probability is close to the true event rate for any given subgroup. Such models are reliable across heterogeneous populations and satisfy strong notions of algorithmic fairness. However, the task of auditing a model for strong calibration is well-known to be difficult -- particularly for machine learning (ML) algorithms -- due to the sheer number of potential subgroups. As such, common practice is to only assess calibration with respect to a few predefined subgroups. Recent developments in goodness-of-fit testing offer potential solutions but are not designed for settings with weak signal or where the poorly calibrated subgroup is small, as they either overly subdivide the data or fail to divide the data at all. We introduce a new testing procedure based on the following insight: if we can reorder observations by their expected residuals, there should be a change in the association between the predicted and observed residuals along this sequence if a poorly calibrated subgroup exists. This lets us reframe the problem of calibration testing into one of changepoint detection, for which powerful methods already exist. We begin with introducing a sample-splitting procedure where a portion of the data is used to train a suite of candidate models for predicting the residual, and the remaining data are used to perform a score-based cumulative sum (CUSUM) test. To further improve power, we then extend this adaptive CUSUM test to incorporate cross-validation, while maintaining Type I error control under minimal assumptions. Compared to existing methods, the proposed procedure consistently achieved higher power in simulation studies and more than doubled the power when auditing a mortality risk prediction model.