# Distribution-Free Statistical Dispersion Control for Societal Applications

arXiv: https://arxiv.org/abs/2309.13786 · HTML: https://mlsanity.com/paper/2309.13786/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [Achieving Equalized Odds by Resampling Sensitive Attributes](https://mlsanity.com/paper/2006.04292/) (2020)
- [Two-sided fairness in rankings via Lorenz dominance](https://mlsanity.com/paper/2110.15781/) (2021)
- [Quantile Risk Control: A Flexible Framework for Bounding the Probability of High-Loss Predictions](https://mlsanity.com/paper/2212.13629/) (2022)
- [Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics](https://mlsanity.com/paper/2202.01615/) (2022)
- [On the exact Berk-Jones statistics and their p-value calculation](https://mlsanity.com/paper/1311.3190/) (2013)

## Built on by (most cited first, 6 of 6)

- [Fast yet Safe: Early-Exiting with Risk Control](https://mlsanity.com/paper/2405.20915/) (2024)
- [Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Models](https://mlsanity.com/paper/2311.13628/) (2023)
- [Quantile Learn-Then-Test: Quantile-Based Risk Control for Hyperparameter Optimization](https://mlsanity.com/paper/2407.17358/) (2024)
- [Improving Predictor Reliability with Selective Recalibration](https://mlsanity.com/paper/2410.05407/) (2024)
- [QuEst: Enhancing Estimates of Quantile-Based Distributional Measures Using Model Predictions](https://mlsanity.com/paper/2507.05220/) (2025)
- [Adversarially Robust Control of Conditional Value-at-Risk via Rockafellar-Uryasev Conformal Inference](https://mlsanity.com/paper/2606.00320/) (2026)

## Abstract (arXiv)

Explicit finite-sample statistical guarantees on model performance are an important ingredient in responsible machine learning. Previous work has focused mainly on bounding either the expected loss of a predictor or the probability that an individual prediction will incur a loss value in a specified range. However, for many high-stakes applications, it is crucial to understand and control the dispersion of a loss distribution, or the extent to which different members of a population experience unequal effects of algorithmic decisions. We initiate the study of distribution-free control of statistical dispersion measures with societal implications and propose a simple yet flexible framework that allows us to handle a much richer class of statistical functionals beyond previous work. Our methods are verified through experiments in toxic comment detection, medical imaging, and film recommendation.