# Gradient Regularized Contrastive Learning for Continual Domain Adaptation

arXiv: https://arxiv.org/abs/2007.12942 · HTML: https://mlsanity.com/paper/2007.12942/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2020.

## Built on by (most cited first, 30 of 35)

- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Domain Adaptation for Time Series Under Feature and Label Shifts](https://mlsanity.com/paper/2302.03133/) (2023)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Layerwise Optimization by Gradient Decomposition for Continual Learning](https://mlsanity.com/paper/2105.07561/) (2021)
- [Revisiting the Transferability of Supervised Pretraining: an MLP Perspective](https://mlsanity.com/paper/2112.00496/) (2021)
- [Contrastive Learning for Unsupervised Domain Adaptation of Time Series](https://mlsanity.com/paper/2206.06243/) (2022)
- [Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation](https://mlsanity.com/paper/2207.10856/) (2022)
- [General Incremental Learning with Domain-aware Categorical Representations](https://mlsanity.com/paper/2204.04078/) (2022)
- [CALDA: Improving Multi-Source Time Series Domain Adaptation with Contrastive Adversarial Learning](https://mlsanity.com/paper/2109.14778/) (2021)
- [Online Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions](https://mlsanity.com/paper/2207.10667/) (2022)
- [To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2307.15063/) (2023)
- [DEJA VU: Continual Model Generalization For Unseen Domains](https://mlsanity.com/paper/2301.10418/) (2023)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [Complementary Domain Adaptation and Generalization for Unsupervised Continual Domain Shift Learning](https://mlsanity.com/paper/2303.15833/) (2023)
- [Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2503.21780/) (2025)
- [Delving into the Continuous Domain Adaptation](https://mlsanity.com/paper/2208.13121/) (2022)
- [Multi-scale Feature Alignment for Continual Learning of Unlabeled Domains](https://mlsanity.com/paper/2302.01287/) (2023)
- [Unsupervised Lifelong Person Re-identification via Contrastive Rehearsal](https://mlsanity.com/paper/2203.06468/) (2022)
- [Anti-Forgetting Adaptation for Unsupervised Person Re-identification](https://mlsanity.com/paper/2411.14695/) (2024)
- [CoSDA: Continual Source-Free Domain Adaptation](https://mlsanity.com/paper/2304.06627/) (2023)
- [Continual Domain Adversarial Adaptation via Double-Head Discriminators](https://mlsanity.com/paper/2402.03588/) (2024)
- [FRIDA -- Generative Feature Replay for Incremental Domain Adaptation](https://mlsanity.com/paper/2112.14316/) (2021)
- [LogoRA: Local-Global Representation Alignment for Robust Time Series Classification](https://mlsanity.com/paper/2409.12169/) (2024)
- [SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operators](https://mlsanity.com/paper/2603.20410/) (2026)
- [Unsupervised domain adaptation via coarse-to-fine feature alignment method using contrastive learning](https://mlsanity.com/paper/2103.12371/) (2021)
- [LLEDA -- Lifelong Self-Supervised Domain Adaptation](https://mlsanity.com/paper/2211.09027/) (2022)
- [Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data](https://mlsanity.com/paper/2103.04789/) (2021)
- [Consecutive Knowledge Meta-Adaptation Learning for Unsupervised Medical Diagnosis](https://mlsanity.com/paper/2209.10425/) (2022)

## Abstract (arXiv)

Human beings can quickly adapt to environmental changes by leveraging learning experience. However, the poor ability of adapting to dynamic environments remains a major challenge for AI models. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labeled source domain and a sequence of unlabeled target domains. There are two major obstacles in this problem: domain shifts and catastrophic forgetting. In this work, we propose Gradient Regularized Contrastive Learning to solve the above obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforces the gradient of contrastive loss not to increase the supervised training loss on the source domain, which maintains the discriminative power of learned features; (2) regularizes the gradient update on the new domain not to increase the classification loss on the old target domains, which enables the model to adapt to an in-coming target domain while preserving the performance of previously observed domains. Hence our method can jointly learn both semantically discriminative and domain-invariant features with labeled source domain and unlabeled target domains. The experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the state-of-the-art.