# Online Continual Adaptation with Active Self-Training

arXiv: https://arxiv.org/abs/2106.06526 · HTML: https://mlsanity.com/paper/2106.06526/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 58 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [Adaptive Gradient-Based Meta-Learning Methods](https://mlsanity.com/paper/1906.02717/) (2019)
- [Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data](https://mlsanity.com/paper/2010.03622/) (2020)
- [Non-stationary Stochastic Optimization](https://mlsanity.com/paper/1307.5449/) (2013)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Online Optimization : Competing with Dynamic Comparators](https://mlsanity.com/paper/1501.06225/) (2015)
- [A Review of Single-Source Deep Unsupervised Visual Domain Adaptation](https://mlsanity.com/paper/2009.00155/) (2020)
- [Active Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.07848/) (2019)
- [Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings](https://mlsanity.com/paper/2010.08666/) (2020)
- [Provable Guarantees for Gradient-Based Meta-Learning](https://mlsanity.com/paper/1902.10644/) (2019)
- [Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient](https://mlsanity.com/paper/1605.04638/) (2016)
- [Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis](https://mlsanity.com/paper/2007.07222/) (2020)
- [A Century of Portraits: A Visual Historical Record of American High School Yearbooks](https://mlsanity.com/paper/1511.02575/) (2015)
- [Incremental Adversarial Domain Adaptation for Continually Changing Environments](https://mlsanity.com/paper/1712.07436/) (2017)
- [ACE: Adapting to Changing Environments for Semantic Segmentation](https://mlsanity.com/paper/1904.06268/) (2019)
- [Dynamical Models and Tracking Regret in Online Convex Programming](https://mlsanity.com/paper/1301.1254/) (2013)
- [Temporal Variability in Implicit Online Learning](https://mlsanity.com/paper/2006.07503/) (2020)
- [Online Adaptive Asymmetric Active Learning with Limited Budgets](https://mlsanity.com/paper/1911.07498/) (2019)

## Built on by (most cited first, 8 of 8)

- [Understanding Gradual Domain Adaptation: Improved Analysis, Optimal Path and Beyond](https://mlsanity.com/paper/2204.08200/) (2022)
- [Adapting to Online Label Shift with Provable Guarantees](https://mlsanity.com/paper/2207.02121/) (2022)
- [Generalizing across Temporal Domains with Koopman Operators](https://mlsanity.com/paper/2402.07834/) (2024)
- [Gradual Domain Adaptation via Normalizing Flows](https://mlsanity.com/paper/2206.11492/) (2022)
- [Cost-effective Framework for Gradual Domain Adaptation with Multifidelity](https://mlsanity.com/paper/2202.04359/) (2022)
- [Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems](https://mlsanity.com/paper/2209.01143/) (2022)
- [Adaptive Shortcut Debiasing for Online Continual Learning](https://mlsanity.com/paper/2312.08677/) (2023)
- [Stable Heterogeneous Treatment Effect Estimation across Out-of-Distribution Populations](https://mlsanity.com/paper/2407.03082/) (2024)

## Abstract (arXiv)

Models trained with offline data often suffer from continual distribution shifts and expensive labeling in changing environments. This calls for a new online learning paradigm where the learner can continually adapt to changing environments with limited labels. In this paper, we propose a new online setting -- Online Active Continual Adaptation, where the learner aims to continually adapt to changing distributions using both unlabeled samples and active queries of limited labels. To this end, we propose Online Self-Adaptive Mirror Descent (OSAMD), which adopts an online teacher-student structure to enable online self-training from unlabeled data, and a margin-based criterion that decides whether to query the labels to track changing distributions. Theoretically, we show that, in the separable case, OSAMD has an O(T2/3) dynamic regret bound under mild assumptions, which is aligned with the Omega(T2/3) lower bound of online learning algorithms with full labels. In the general case, we show a regret bound of O(T2/3 + α^* T), where α^* denotes the separability of domains and is usually small. Our theoretical results show that OSAMD can fast adapt to changing environments with active queries. Empirically, we demonstrate that OSAMD achieves favorable regrets under changing environments with limited labels on both simulated and real-world data, which corroborates our theoretical findings.