# Gradual Domain Adaptation: Theory and Algorithms

arXiv: https://arxiv.org/abs/2310.13852 · HTML: https://mlsanity.com/paper/2310.13852/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 77 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 30 of 39)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks](https://mlsanity.com/paper/1905.13210/) (2019)
- [DLOW: Domain Flow for Adaptation and Generalization](https://mlsanity.com/paper/1812.05418/) (2018)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift](https://mlsanity.com/paper/2003.04475/) (2020)
- [FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2011.09230/) (2020)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Extending the WILDS Benchmark for Unsupervised Adaptation](https://mlsanity.com/paper/2112.05090/) (2021)
- [Domain-Adversarial Neural Networks](https://mlsanity.com/paper/1412.4446/) (2014)
- [Continuously Indexed Domain Adaptation](https://mlsanity.com/paper/2007.01807/) (2020)

## Built on by (most cited first, 18 of 18)

- [Pave Your Own Path: Graph Gradual Domain Adaptation on Fused Gromov-Wasserstein Geodesics](https://mlsanity.com/paper/2505.12709/) (2025)
- [FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2510.20250/) (2025)
- [Gradual Domain Adaptation via Manifold-Constrained Distributionally Robust Optimization](https://mlsanity.com/paper/2410.14061/) (2024)
- [Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalization](https://mlsanity.com/paper/2412.05169/) (2024)
- [Zero Shot Domain Adaptive Semantic Segmentation by Synthetic Data Generation and Progressive Adaptation](https://mlsanity.com/paper/2508.03300/) (2025)
- [Gradual Domain Adaptation for Graph Learning](https://mlsanity.com/paper/2501.17443/) (2025)
- [Adapting to Evolving Adversaries with Regularized Continual Robust Training](https://mlsanity.com/paper/2502.04248/) (2025)
- [Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation](https://mlsanity.com/paper/2510.13864/) (2025)
- [Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift](https://mlsanity.com/paper/2602.14913/) (2026)
- [SWAT: Sliding Window Adversarial Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2501.19155/) (2025)
- [GDO:Gradual Domain Osmosis](https://mlsanity.com/paper/2501.19159/) (2025)
- [Autonomous Source Knowledge Selection in Multi-Domain Adaptation](https://mlsanity.com/paper/2512.14710/) (2025)
- [Gradual Fine-Tuning for Flow Matching Models](https://mlsanity.com/paper/2601.22495/) (2026)
- [Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspective](https://mlsanity.com/paper/2602.01179/) (2026)
- [Rationality Measurement and Theory for Reinforcement Learning Agents](https://mlsanity.com/paper/2602.04737/) (2026)
- [Multi-Scale Temporal Homeostasis Enables Efficient and Robust Neural Networks](https://mlsanity.com/paper/2602.07009/) (2026)
- [Transfer learning via interpolating structures](https://mlsanity.com/paper/2603.22621/) (2026)
- [Tunable Domain Adaptation Using Unfolding](https://mlsanity.com/paper/2603.26931/) (2026)

## Abstract (arXiv)

Unsupervised domain adaptation (UDA) adapts a model from a labeled source domain to an unlabeled target domain in a one-off way. Though widely applied, UDA faces a great challenge whenever the distribution shift between the source and the target is large. Gradual domain adaptation (GDA) mitigates this limitation by using intermediate domains to gradually adapt from the source to the target domain. In this work, we first theoretically analyze gradual self-training, a popular GDA algorithm, and provide a significantly improved generalization bound compared with Kumar et al. (2020). Our theoretical analysis leads to an interesting insight: to minimize the generalization error on the target domain, the sequence of intermediate domains should be placed uniformly along the Wasserstein geodesic between the source and target domains. The insight is particularly useful under the situation where intermediate domains are missing or scarce, which is often the case in real-world applications. Based on the insight, we propose Generative Gradual DOmain Adaptation with Optimal Transport (GOAT), an algorithmic framework that can generate intermediate domains in a data-dependent way. More concretely, we first generate intermediate domains along the Wasserstein geodesic between two given consecutive domains in a feature space, then apply gradual self-training to adapt the source-trained classifier to the target along the sequence of intermediate domains. Empirically, we demonstrate that our GOAT framework can improve the performance of standard GDA when the given intermediate domains are scarce, significantly broadening the real-world application scenarios of GDA. Our code is available at https://github.com/uiuctml/GOAT.