# Stable Prediction with Model Misspecification and Agnostic Distribution Shift

arXiv: https://arxiv.org/abs/2001.11713 · HTML: https://mlsanity.com/paper/2001.11713/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2020.

## Built on by (most cited first, 30 of 62)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [Heterogeneous Risk Minimization](https://mlsanity.com/paper/2105.03818/) (2021)
- [OOD-GNN: Out-of-Distribution Generalized Graph Neural Network](https://mlsanity.com/paper/2112.03806/) (2021)
- [Generalizing Graph Neural Networks on Out-Of-Distribution Graphs](https://mlsanity.com/paper/2111.10657/) (2021)
- [MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning](https://mlsanity.com/paper/2310.05506/) (2023)
- [Model Agnostic Sample Reweighting for Out-of-Distribution Learning](https://mlsanity.com/paper/2301.09819/) (2023)
- [NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02038/) (2021)
- [Debiased Graph Neural Networks with Agnostic Label Selection Bias](https://mlsanity.com/paper/2201.07708/) (2022)
- [Kernelized Heterogeneous Risk Minimization](https://mlsanity.com/paper/2110.12425/) (2021)
- [A Survey on Evaluation of Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.01874/) (2024)
- [Rethinking the Evaluation Protocol of Domain Generalization](https://mlsanity.com/paper/2305.15253/) (2023)
- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [Exploring Optimal Substructure for Out-of-distribution Generalization via Feature-targeted Model Pruning](https://mlsanity.com/paper/2212.09458/) (2022)
- [Towards Domain Generalization in Object Detection](https://mlsanity.com/paper/2203.14387/) (2022)
- [Label-Efficient Domain Generalization via Collaborative Exploration and Generalization](https://mlsanity.com/paper/2208.03644/) (2022)
- [Distributionally Robust Learning with Stable Adversarial Training](https://mlsanity.com/paper/2106.15791/) (2021)
- [Stable Learning via Sparse Variable Independence](https://mlsanity.com/paper/2212.00992/) (2022)
- [Out-of-Distribution Generalization Analysis via Influence Function](https://mlsanity.com/paper/2101.08521/) (2021)
- [Towards out of distribution generalization for problems in mechanics](https://mlsanity.com/paper/2206.14917/) (2022)
- [ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations](https://mlsanity.com/paper/2303.01092/) (2023)
- [Stable Prediction on Graphs with Agnostic Distribution Shift](https://mlsanity.com/paper/2110.03865/) (2021)
- [SFP: Spurious Feature-targeted Pruning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2305.11615/) (2023)
- [Grounded Image Text Matching with Mismatched Relation Reasoning](https://mlsanity.com/paper/2308.01236/) (2023)
- [General Greedy De-bias Learning](https://mlsanity.com/paper/2112.10572/) (2021)
- [Learning Stable Classifiers by Transferring Unstable Features](https://mlsanity.com/paper/2106.07847/) (2021)
- [Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds](https://mlsanity.com/paper/2202.13524/) (2022)
- [RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring](https://mlsanity.com/paper/2206.00568/) (2022)
- [Mining Stable Preferences: Adaptive Modality Decorrelation for Multimedia Recommendation](https://mlsanity.com/paper/2306.14179/) (2023)
- [Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness](https://mlsanity.com/paper/2509.17228/) (2025)

## Abstract (arXiv)

For many machine learning algorithms, two main assumptions are required to guarantee performance. One is that the test data are drawn from the same distribution as the training data, and the other is that the model is correctly specified. In real applications, however, we often have little prior knowledge on the test data and on the underlying true model. Under model misspecification, agnostic distribution shift between training and test data leads to inaccuracy of parameter estimation and instability of prediction across unknown test data. To address these problems, we propose a novel Decorrelated Weighting Regression (DWR) algorithm which jointly optimizes a variable decorrelation regularizer and a weighted regression model. The variable decorrelation regularizer estimates a weight for each sample such that variables are decorrelated on the weighted training data. Then, these weights are used in the weighted regression to improve the accuracy of estimation on the effect of each variable, thus help to improve the stability of prediction across unknown test data. Extensive experiments clearly demonstrate that our DWR algorithm can significantly improve the accuracy of parameter estimation and stability of prediction with model misspecification and agnostic distribution shift.