# Understanding Gradual Domain Adaptation: Improved Analysis, Optimal Path and Beyond

arXiv: https://arxiv.org/abs/2204.08200 · HTML: https://mlsanity.com/paper/2204.08200/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 78 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks](https://mlsanity.com/paper/1905.13210/) (2019)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks](https://mlsanity.com/paper/2107.07455/) (2021)
- [Extending the WILDS Benchmark for Unsupervised Adaptation](https://mlsanity.com/paper/2112.05090/) (2021)
- [Domain-Adversarial Neural Networks](https://mlsanity.com/paper/1412.4446/) (2014)
- [Continuously Indexed Domain Adaptation](https://mlsanity.com/paper/2007.01807/) (2020)
- [Optimal Transport for Multi-source Domain Adaptation under Target Shift](https://mlsanity.com/paper/1803.04899/) (2018)
- [A Century of Portraits: A Visual Historical Record of American High School Yearbooks](https://mlsanity.com/paper/1511.02575/) (2015)
- [Incremental Adversarial Domain Adaptation for Continually Changing Environments](https://mlsanity.com/paper/1712.07436/) (2017)
- [Adversarial Domain Adaptation for Stable Brain-Machine Interfaces](https://mlsanity.com/paper/1810.00045/) (2018)
- [Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent](https://mlsanity.com/paper/2106.06080/) (2021)

## Built on by (most cited first, 30 of 32)

- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation](https://mlsanity.com/paper/2210.10195/) (2022)
- [Gradual Domain Adaptation: Theory and Algorithms](https://mlsanity.com/paper/2310.13852/) (2023)
- [Pave Your Own Path: Graph Gradual Domain Adaptation on Fused Gromov-Wasserstein Geodesics](https://mlsanity.com/paper/2505.12709/) (2025)
- [Generalizing across Temporal Domains with Koopman Operators](https://mlsanity.com/paper/2402.07834/) (2024)
- [Gradual Domain Adaptation via Normalizing Flows](https://mlsanity.com/paper/2206.11492/) (2022)
- [Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos](https://mlsanity.com/paper/2311.16444/) (2023)
- [Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift](https://mlsanity.com/paper/2406.00661/) (2024)
- [HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation](https://mlsanity.com/paper/2403.16788/) (2024)
- [EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs](https://mlsanity.com/paper/2305.00664/) (2023)
- [Temporal Test-Time Adaptation with State-Space Models](https://mlsanity.com/paper/2407.12492/) (2024)
- [Enhancing Evolving Domain Generalization through Dynamic Latent Representations](https://mlsanity.com/paper/2401.08464/) (2024)
- [Progressive Multi-Source Domain Adaptation for Personalized Facial Expression Recognition](https://mlsanity.com/paper/2504.04252/) (2025)
- [Evolving Domain Adaptation of Pretrained Language Models for Text Classification](https://mlsanity.com/paper/2311.09661/) (2023)
- [Algorithms and Theory for Supervised Gradual Domain Adaptation](https://mlsanity.com/paper/2204.11644/) (2022)
- [Continual Domain Adversarial Adaptation via Double-Head Discriminators](https://mlsanity.com/paper/2402.03588/) (2024)
- [Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems](https://mlsanity.com/paper/2209.01143/) (2022)
- [Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics](https://mlsanity.com/paper/2302.02092/) (2023)
- [Gradual Domain Adaptation via Manifold-Constrained Distributionally Robust Optimization](https://mlsanity.com/paper/2410.14061/) (2024)
- [Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalization](https://mlsanity.com/paper/2412.05169/) (2024)
- [Gradual Domain Adaptation for Graph Learning](https://mlsanity.com/paper/2501.17443/) (2025)
- [Adapting to Evolving Adversaries with Regularized Continual Robust Training](https://mlsanity.com/paper/2502.04248/) (2025)
- [Invariant-Feature Subspace Recovery: A New Class of Provable Domain Generalization Algorithms](https://mlsanity.com/paper/2311.00966/) (2023)
- [Bidirectional Multi-Step Domain Generalization for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2403.10782/) (2024)
- [Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention](https://mlsanity.com/paper/2505.07023/) (2025)
- [Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation](https://mlsanity.com/paper/2510.13864/) (2025)
- [Gradual Fine-Tuning with Graph Routing for Multi-Source Unsupervised Domain Adaptation](https://mlsanity.com/paper/2411.07185/) (2024)
- [GDO:Gradual Domain Osmosis](https://mlsanity.com/paper/2501.19159/) (2025)
- [Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspective](https://mlsanity.com/paper/2602.01179/) (2026)
- [Transfer learning via interpolating structures](https://mlsanity.com/paper/2603.22621/) (2026)

## Abstract (arXiv)

The vast majority of existing algorithms for unsupervised domain adaptation (UDA) focus on adapting from a labeled source domain to an unlabeled target domain directly in a one-off way. Gradual domain adaptation (GDA), on the other hand, assumes a path of (T-1) unlabeled intermediate domains bridging the source and target, and aims to provide better generalization in the target domain by leveraging the intermediate ones. Under certain assumptions, Kumar et al. (2020) proposed a simple algorithm, Gradual Self-Training, along with a generalization bound in the order of eO(T) left(varepsilon0+Oleft(√log(T)/nright)right) for the target domain error, where varepsilon0 is the source domain error and n is the data size of each domain. Due to the exponential factor, this upper bound becomes vacuous when T is only moderately large. In this work, we analyze gradual self-training under more general and relaxed assumptions, and prove a significantly improved generalization bound as varepsilon0+ O left(TDelta + T/√nright) + widetildeOleft(1/√nTright), where Delta is the average distributional distance between consecutive domains. Compared with the existing bound with an exponential dependency on T as a multiplicative factor, our bound only depends on T linearly and additively. Perhaps more interestingly, our result implies the existence of an optimal choice of T that minimizes the generalization error, and it also naturally suggests an optimal way to construct the path of intermediate domains so as to minimize the accumulative path length TDelta between the source and target. To corroborate the implications of our theory, we examine gradual self-training on multiple semi-synthetic and real datasets, which confirms our findings. We believe our insights provide a path forward toward the design of future GDA algorithms.