# Provable Detection of Propagating Sampling Bias in Prediction Models

arXiv: https://arxiv.org/abs/2302.06752 · HTML: https://mlsanity.com/paper/2302.06752/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 13 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [Algorithmic Fairness](https://mlsanity.com/paper/2001.09784/) (2020)
- [Learning Controllable Fair Representations](https://mlsanity.com/paper/1812.04218/) (2018)
- [On Formalizing Fairness in Prediction with Machine Learning](https://mlsanity.com/paper/1710.03184/) (2017)
- [ProPublica's COMPAS Data Revisited](https://mlsanity.com/paper/1906.04711/) (2019)
- [Identifying Significant Predictive Bias in Classifiers](https://mlsanity.com/paper/1611.08292/) (2016)
- [Bias In, Bias Out? Evaluating the Folk Wisdom](https://mlsanity.com/paper/1909.08518/) (2019)
- [A Causal Approach for Unfair Edge Prioritization and Discrimination Removal](https://mlsanity.com/paper/2111.14348/) (2021)

## Built on by (most cited first, 1 of 1)

- [Learning Representational Disparities](https://mlsanity.com/paper/2505.17533/) (2025)

## Abstract (arXiv)

With an increased focus on incorporating fairness in machine learning models, it becomes imperative not only to assess and mitigate bias at each stage of the machine learning pipeline but also to understand the downstream impacts of bias across stages. Here we consider a general, but realistic, scenario in which a predictive model is learned from (potentially biased) training data, and model predictions are assessed post-hoc for fairness by some auditing method. We provide a theoretical analysis of how a specific form of data bias, differential sampling bias, propagates from the data stage to the prediction stage. Unlike prior work, we evaluate the downstream impacts of data biases quantitatively rather than qualitatively and prove theoretical guarantees for detection. Under reasonable assumptions, we quantify how the amount of bias in the model predictions varies as a function of the amount of differential sampling bias in the data, and at what point this bias becomes provably detectable by the auditor. Through experiments on two criminal justice datasets -- the well-known COMPAS dataset and historical data from NYPD's stop and frisk policy -- we demonstrate that the theoretical results hold in practice even when our assumptions are relaxed.