# Weighted Empirical Risk Minimization: Sample Selection Bias Correction based on Importance Sampling

arXiv: https://arxiv.org/abs/2002.05145 · HTML: https://mlsanity.com/paper/2002.05145/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [Strong Model Collapse](https://mlsanity.com/paper/2410.04840/) (2024)
- [FairDD: Fair Dataset Distillation](https://mlsanity.com/paper/2411.19623/) (2024)
- [Safety Performance of Neural Networks in the Presence of Covariate Shift](https://mlsanity.com/paper/2307.12716/) (2023)
- [Fast Rates for Nonstationary Weighted Risk Minimization](https://mlsanity.com/paper/2602.05742/) (2026)
- [Fighting Selection Bias in Statistical Learning: Application to Visual Recognition from Biased Image Databases](https://mlsanity.com/paper/2109.02357/) (2021)
- [Sample Selection Bias in Machine Learning for Healthcare](https://mlsanity.com/paper/2405.07841/) (2024)
- [Mitigating the Participation Bias by Balancing Extreme Ratings](https://mlsanity.com/paper/2502.03737/) (2025)

## Abstract (arXiv)

We consider statistical learning problems, when the distribution P' of the training observations Z'1,; ldots,; Z'n differs from the distribution P involved in the risk one seeks to minimize (referred to as the test distribution) but is still defined on the same measurable space as P and dominates it. In the unrealistic case where the likelihood ratio Phi(z)=dP/dP'(z) is known, one may straightforwardly extends the Empirical Risk Minimization (ERM) approach to this specific transfer learning setup using the same idea as that behind Importance Sampling, by minimizing a weighted version of the empirical risk functional computed from the 'biased' training data Z'i with weights Phi(Z'i). Although the importance function Phi(z) is generally unknown in practice, we show that, in various situations frequently encountered in practice, it takes a simple form and can be directly estimated from the Z'i's and some auxiliary information on the statistical population P. By means of linearization techniques, we then prove that the generalization capacity of the approach aforementioned is preserved when plugging the resulting estimates of the Phi(Z'i)'s into the weighted empirical risk. Beyond these theoretical guarantees, numerical results provide strong empirical evidence of the relevance of the approach promoted in this article.