# Stable Learning via Sample Reweighting

arXiv: https://arxiv.org/abs/1911.12580 · HTML: https://mlsanity.com/paper/1911.12580/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2019.

## Built on by (most cited first, 30 of 70)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [OOD-GNN: Out-of-Distribution Generalized Graph Neural Network](https://mlsanity.com/paper/2112.03806/) (2021)
- [Generalizing Graph Neural Networks on Out-Of-Distribution Graphs](https://mlsanity.com/paper/2111.10657/) (2021)
- [Out-of-distribution Generalization with Causal Invariant Transformations](https://mlsanity.com/paper/2203.11528/) (2022)
- [MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning](https://mlsanity.com/paper/2310.05506/) (2023)
- [Model Agnostic Sample Reweighting for Out-of-Distribution Learning](https://mlsanity.com/paper/2301.09819/) (2023)
- [Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization](https://mlsanity.com/paper/2306.01103/) (2023)
- [Debiased Graph Neural Networks with Agnostic Label Selection Bias](https://mlsanity.com/paper/2201.07708/) (2022)
- [A Survey on Evaluation of Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.01874/) (2024)
- [Rethinking the Evaluation Protocol of Domain Generalization](https://mlsanity.com/paper/2305.15253/) (2023)
- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [Exploring Optimal Substructure for Out-of-distribution Generalization via Feature-targeted Model Pruning](https://mlsanity.com/paper/2212.09458/) (2022)
- [Towards Domain Generalization in Object Detection](https://mlsanity.com/paper/2203.14387/) (2022)
- [Label-Efficient Domain Generalization via Collaborative Exploration and Generalization](https://mlsanity.com/paper/2208.03644/) (2022)
- [Distributionally Robust Learning with Stable Adversarial Training](https://mlsanity.com/paper/2106.15791/) (2021)
- [Stable Learning via Sparse Variable Independence](https://mlsanity.com/paper/2212.00992/) (2022)
- [ZooD: Exploiting Model Zoo for Out-of-Distribution Generalization](https://mlsanity.com/paper/2210.09236/) (2022)
- [Instrumental Variables in Causal Inference and Machine Learning: A Survey](https://mlsanity.com/paper/2212.05778/) (2022)
- [ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations](https://mlsanity.com/paper/2303.01092/) (2023)
- [Graph Structure and Feature Extrapolation for Out-of-Distribution Generalization](https://mlsanity.com/paper/2306.08076/) (2023)
- [SFP: Spurious Feature-targeted Pruning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2305.11615/) (2023)
- [Grounded Image Text Matching with Mismatched Relation Reasoning](https://mlsanity.com/paper/2308.01236/) (2023)
- [General Greedy De-bias Learning](https://mlsanity.com/paper/2112.10572/) (2021)
- [Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data](https://mlsanity.com/paper/2111.09190/) (2021)
- [Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds](https://mlsanity.com/paper/2202.13524/) (2022)
- [RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring](https://mlsanity.com/paper/2206.00568/) (2022)
- [PearNet: A Pearson Correlation-based Graph Attention Network for Sleep Stage Recognition](https://mlsanity.com/paper/2209.13645/) (2022)
- [Mining Stable Preferences: Adaptive Modality Decorrelation for Multimedia Recommendation](https://mlsanity.com/paper/2306.14179/) (2023)
- [Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective](https://mlsanity.com/paper/2202.08048/) (2022)

## Abstract (arXiv)

We consider the problem of learning linear prediction models with model misspecification bias. In such case, the collinearity among input variables may inflate the error of parameter estimation, resulting in instability of prediction results when training and test distributions do not match. In this paper we theoretically analyze this fundamental problem and propose a sample reweighting method that reduces collinearity among input variables. Our method can be seen as a pretreatment of data to improve the condition of design matrix, and it can then be combined with any standard learning method for parameter estimation and variable selection. Empirical studies on both simulation and real datasets demonstrate the effectiveness of our method in terms of more stable performance across different distributed data.