mlsanityData to 7 Oct 2026

Understanding Self-Training for Gradual Domain Adaptation

arXiv 2002.11361 ↗First posted 26 Feb 2020cs.LG stat.MLcited by 149

Not on the map. Why?

Abstract (arXiv)

Machine learning systems must adapt to data distributions that evolve over time, in applications ranging from sensor networks and self-driving car perception modules to brain-machine interfaces. We consider gradual domain adaptation, where the goal is to adapt an initial classifier trained on a source domain given only unlabeled data that shifts gradually in distribution towards a target domain. We prove the first non-vacuous upper bound on the error of self-training with gradual shifts, under settings where directly adapting to the target domain can result in unbounded error. The theoretical analysis leads to algorithmic insights, highlighting that regularization and label sharpening are essential even when we have infinite data, and suggesting that self-training works particularly well for shifts with small Wasserstein-infinity distance. Leveraging the gradual shift structure leads to higher accuracies on a rotating MNIST dataset and a realistic Portraits dataset.

Key facts. Cited by 149 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 36; 2023: 37; 2024: 21; 2025: 18; 2026: 13.

149 papers.

Sort:

By area:

  1. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  2. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  3. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  4. 2021ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentationcited by 184
  5. 2021Cycle Self-Training for Domain Adaptationcited by 120
  6. 2022Robust Mean Teacher for Continual and Gradual Test-Time Adaptationcited by 102
  7. 2022Test-Time Adaptation via Conjugate Pseudo-labelscited by 98
  8. 2022Self-Training: A Surveycited by 85
  9. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  10. 2021Break-It-Fix-It: Unsupervised Learning for Program Repaircited by 71
  11. 2022Back to the Source: Diffusion-Driven Test-Time Adaptationcited by 71
  12. 2022Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptationcited by 66
38 more
  1. 2021Editing a classifier by rewriting its prediction rulescited by 64
  2. 2023Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detectioncited by 64
  3. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 59
  4. 2023Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detectioncited by 57
  5. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
  6. 2021A Theory of Label Propagation for Subpopulation Shiftcited by 48
  7. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 46
  8. 2022Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulationscited by 44
  9. 2021Generate, Annotate, and Learn: NLP with Synthetic Textcited by 42
  10. 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
  11. 2022SuperAnimal pretrained pose estimation models for behavioral analysiscited by 38
  12. 2023Test-Time Training on Video Streamscited by 37
  13. 2021If your data distribution shifts, use self-learningcited by 36
  14. 2023Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalizationcited by 34
  15. 2022Understanding Gradual Domain Adaptation: Improved Analysis, Optimal Path and Beyondcited by 32
  16. 2023RLSbench: Domain Adaptation Under Relaxed Label Shiftcited by 30
  17. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  18. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  19. 2021Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Timecited by 27
  20. 2021Multimodal Knowledge Expansioncited by 26
  21. 2022Gradual Domain Adaptation without Indexed Intermediate Domainscited by 26
  22. 2022On the Domain Adaptation and Generalization of Pretrained Language Models: A Surveycited by 22
  23. 2021Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detectioncited by 21
  24. 2021Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absentcited by 21
  25. 2022Unknown-Aware Domain Adversarial Learning for Open-Set Domain Adaptationcited by 21
  26. 2022Transferring Fairness under Distribution Shifts via Fair Consistency Regularizationcited by 21
  27. 2021Question Answering Infused Pre-training of General-Purpose Contextualized Representationscited by 19
  28. 2021Self-training Converts Weak Learners to Strong Learners in Mixture Modelscited by 19
  29. 2023Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptationcited by 19
  30. 2023Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Trainingcited by 19
  31. 2023Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!cited by 19
  32. 2022Generalizing to Evolving Domains with Latent Structure-Aware Sequential Autoencodercited by 18
  33. 2023Gradual Domain Adaptation: Theory and Algorithmscited by 18
  34. 2023Pseudo-Labeling for Kernel Ridge Regression under Covariate Shiftcited by 17
  35. 2025Pave Your Own Path: Graph Gradual Domain Adaptation on Fused Gromov-Wasserstein Geodesicscited by 17
  36. 2023Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?cited by 14
  37. 2023Unified Out-Of-Distribution Detection: A Model-Specific Perspectivecited by 13
  38. 2021Combining Diverse Feature Priorscited by 12

50 most cited of 149 All 149 (JSON)