# Stable Prediction across Unknown Environments

arXiv: https://arxiv.org/abs/1806.06270 · HTML: https://mlsanity.com/paper/1806.06270/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2018.

## Built on by (most cited first, 30 of 55)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [Out-Of-Distribution Generalization on Graphs: A Survey](https://mlsanity.com/paper/2202.07987/) (2022)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [OOD-GNN: Out-of-Distribution Generalized Graph Neural Network](https://mlsanity.com/paper/2112.03806/) (2021)
- [Generalizing Graph Neural Networks on Out-Of-Distribution Graphs](https://mlsanity.com/paper/2111.10657/) (2021)
- [Out-of-distribution Generalization with Causal Invariant Transformations](https://mlsanity.com/paper/2203.11528/) (2022)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02038/) (2021)
- [Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix](https://mlsanity.com/paper/2101.07732/) (2021)
- [Debiased Graph Neural Networks with Agnostic Label Selection Bias](https://mlsanity.com/paper/2201.07708/) (2022)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [Auto IV: Counterfactual Prediction via Automatic Instrumental Variable Decomposition](https://mlsanity.com/paper/2107.05884/) (2021)
- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Optimization-based Causal Estimation from Heterogenous Environments](https://mlsanity.com/paper/2109.11990/) (2021)
- [Label-Efficient Domain Generalization via Collaborative Exploration and Generalization](https://mlsanity.com/paper/2208.03644/) (2022)
- [Distributionally Robust Learning with Stable Adversarial Training](https://mlsanity.com/paper/2106.15791/) (2021)
- [Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization](https://mlsanity.com/paper/2306.02595/) (2023)
- [Stable Learning via Sparse Variable Independence](https://mlsanity.com/paper/2212.00992/) (2022)
- [ZooD: Exploiting Model Zoo for Out-of-Distribution Generalization](https://mlsanity.com/paper/2210.09236/) (2022)
- [Stable Prediction on Graphs with Agnostic Distribution Shift](https://mlsanity.com/paper/2110.03865/) (2021)
- [Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods](https://mlsanity.com/paper/2308.06703/) (2023)
- [Robust Generalization despite Distribution Shift via Minimum Discriminating Information](https://mlsanity.com/paper/2106.04443/) (2021)
- [CausalTAD: Causal Implicit Generative Model for Debiased Online Trajectory Anomaly Detection](https://mlsanity.com/paper/2412.18820/) (2024)
- [RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring](https://mlsanity.com/paper/2206.00568/) (2022)
- [Mining Stable Preferences: Adaptive Modality Decorrelation for Multimedia Recommendation](https://mlsanity.com/paper/2306.14179/) (2023)
- [AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge Distillation](https://mlsanity.com/paper/2403.07030/) (2024)
- [MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation](https://mlsanity.com/paper/2404.19644/) (2024)

## Abstract (arXiv)

In many important machine learning applications, the training distribution used to learn a probabilistic classifier differs from the testing distribution on which the classifier will be used to make predictions. Traditional methods correct the distribution shift by reweighting the training data with the ratio of the density between test and training data. In many applications training takes place without prior knowledge of the testing distribution on which the algorithm will be applied in the future. Recently, methods have been proposed to address the shift by learning causal structure, but those methods rely on the diversity of multiple training data to a good performance, and have complexity limitations in high dimensions. In this paper, we propose a novel Deep Global Balancing Regression (DGBR) algorithm to jointly optimize a deep auto-encoder model for feature selection and a global balancing model for stable prediction across unknown environments. The global balancing model constructs balancing weights that facilitate estimating of partial effects of features (holding fixed all other features), a problem that is challenging in high dimensions, and thus helps to identify stable, causal relationships between features and outcomes. The deep auto-encoder model is designed to reduce the dimensionality of the feature space, thus making global balancing easier. We show, both theoretically and with empirical experiments, that our algorithm can make stable predictions across unknown environments. Our experiments on both synthetic and real world datasets demonstrate that our DGBR algorithm outperforms the state-of-the-art methods for stable prediction across unknown environments.