# A Theory of Label Propagation for Subpopulation Shift

arXiv: https://arxiv.org/abs/2102.11203 · HTML: https://mlsanity.com/paper/2102.11203/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 85 references that are in the map.
- First posted on arXiv 22 Feb 2021.

## Builds on (most cited first, 30 of 49)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1903.04064/) (2019)
- [How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks](https://mlsanity.com/paper/2009.11848/) (2020)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [Multi-Adversarial Domain Adaptation](https://mlsanity.com/paper/1809.02176/) (2018)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data](https://mlsanity.com/paper/2010.03622/) (2020)
- [BREEDS: Benchmarks for Subpopulation Shift](https://mlsanity.com/paper/2008.04859/) (2020)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Deep Co-Training for Semi-Supervised Image Recognition](https://mlsanity.com/paper/1803.05984/) (2018)

## Built on by (most cited first, 30 of 48)

- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Cycle Self-Training for Domain Adaptation](https://mlsanity.com/paper/2103.03571/) (2021)
- [Towards a Theoretical Framework of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.04496/) (2021)
- [Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2204.00570/) (2022)
- [Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning](https://mlsanity.com/paper/2211.06612/) (2022)
- [Theoretical Analysis of Weak-to-Strong Generalization](https://mlsanity.com/paper/2405.16043/) (2024)
- [Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations](https://mlsanity.com/paper/2204.02683/) (2022)
- [Near-Optimal Linear Regression under Distribution Shift](https://mlsanity.com/paper/2106.12108/) (2021)
- [Invariant and Transportable Representations for Anti-Causal Domain Shifts](https://mlsanity.com/paper/2207.01603/) (2022)
- [If your data distribution shifts, use self-learning](https://mlsanity.com/paper/2104.12928/) (2021)
- [Sample Efficiency of Data Augmentation Consistency Regularization](https://mlsanity.com/paper/2202.12230/) (2022)
- [Transferring Fairness under Distribution Shifts via Fair Consistency Regularization](https://mlsanity.com/paper/2206.12796/) (2022)
- [First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains](https://mlsanity.com/paper/2211.11719/) (2022)
- [Self-training Converts Weak Learners to Strong Learners in Mixture Models](https://mlsanity.com/paper/2106.13805/) (2021)
- [Localized Adversarial Domain Generalization](https://mlsanity.com/paper/2205.04114/) (2022)
- [Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift](https://mlsanity.com/paper/2302.10160/) (2023)
- [GeoNet: Benchmarking Unsupervised Adaptation across Geographies](https://mlsanity.com/paper/2303.15443/) (2023)
- [The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment](https://mlsanity.com/paper/2412.16468/) (2024)
- [Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimension](https://mlsanity.com/paper/2502.05075/) (2025)
- [Understanding new tasks through the lens of training data via exponential tilting](https://mlsanity.com/paper/2205.13577/) (2022)
- [Controllable Prompt Tuning For Balancing Group Distributional Robustness](https://mlsanity.com/paper/2403.02695/) (2024)
- [Label Propagation with Weak Supervision](https://mlsanity.com/paper/2210.03594/) (2022)
- [Can domain adaptation make object recognition work for everyone?](https://mlsanity.com/paper/2204.11122/) (2022)
- [Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets](https://mlsanity.com/paper/2510.02818/) (2025)
- [Cluster-aware Semi-supervised Learning: Relational Knowledge Distillation Provably Learns Clustering](https://mlsanity.com/paper/2307.11030/) (2023)
- [Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift](https://mlsanity.com/paper/2312.03318/) (2023)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)
- [Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift](https://mlsanity.com/paper/2310.04971/) (2023)
- [Analyzing Similarity Metrics for Data Selection for Language Model Pretraining](https://mlsanity.com/paper/2502.02494/) (2025)

## Abstract (arXiv)

One of the central problems in machine learning is domain adaptation. Unlike past theoretical work, we consider a new model for subpopulation shift in the input or representation space. In this work, we propose a provably effective framework for domain adaptation based on label propagation. In our analysis, we use a simple but realistic expansion assumption, proposed in citetwei2021theoretical. Using a teacher classifier trained on the source domain, our algorithm not only propagates to the target domain but also improves upon the teacher. By leveraging existing generalization bounds, we also obtain end-to-end finite-sample guarantees on the entire algorithm. In addition, we extend our theoretical framework to a more general setting of source-to-target transfer based on a third unlabeled dataset, which can be easily applied in various learning scenarios. Inspired by our theory, we adapt consistency-based semi-supervised learning methods to domain adaptation settings and gain significant improvements.