# Statistical Learning from Biased Training Samples

arXiv: https://arxiv.org/abs/1906.12304 · HTML: https://mlsanity.com/paper/1906.12304/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2019.

## Built on by (most cited first, 4 of 4)

- [Robust Fairness-aware Learning Under Sample Selection Bias](https://mlsanity.com/paper/2105.11570/) (2021)
- [Fair Regression under Sample Selection Bias](https://mlsanity.com/paper/2110.04372/) (2021)
- [Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classification](https://mlsanity.com/paper/2404.10757/) (2024)
- [Fighting Selection Bias in Statistical Learning: Application to Visual Recognition from Biased Image Databases](https://mlsanity.com/paper/2109.02357/) (2021)

## Abstract (arXiv)

With the deluge of digitized information in the Big Data era, massive datasets are becoming increasingly available for learning predictive models. However, in many practical situations, the poor control of the data acquisition processes may naturally jeopardize the outputs of machine learning algorithms, and selection bias issues are now the subject of much attention in the literature. The present article investigates how to extend Empirical Risk Minimization, the principal paradigm in statistical learning, when training observations are generated from biased models, i.e., from distributions that are different from that in the test/prediction stage, and absolutely continuous with respect to the latter. Precisely, we show how to build a "nearly debiased" training statistical population from biased samples and the related biasing functions, following in the footsteps of the approach originally proposed in Vardi (1985). Furthermore, we study from a nonasymptotic perspective the performance of minimizers of an empirical version of the risk computed from the statistical population thus created. Remarkably, the learning rate achieved by this procedure is of the same order as that attained in absence of selection bias. Beyond the theoretical guarantees, we also present experimental results supporting the relevance of the algorithmic approach promoted in this paper.