# Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation

arXiv: https://arxiv.org/abs/2304.07560 · HTML: https://mlsanity.com/paper/2304.07560/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 33 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [Instance Adaptive Self-Training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.12197/) (2020)
- [Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation](https://mlsanity.com/paper/2110.04202/) (2021)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer](https://mlsanity.com/paper/2012.07297/) (2020)
- [Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/1910.13049/) (2019)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)
- [A Closer Look at Smoothness in Domain Adversarial Training](https://mlsanity.com/paper/2206.08213/) (2022)
- [CHIP: CHannel Independence-based Pruning for Compact Neural Networks](https://mlsanity.com/paper/2110.13981/) (2021)
- [Adaptive Methods for Real-World Domain Generalization](https://mlsanity.com/paper/2103.15796/) (2021)
- [Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings](https://mlsanity.com/paper/2011.12616/) (2020)
- [CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.10464/) (2020)
- [PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations](https://mlsanity.com/paper/2203.16965/) (2022)
- [DSD: Dense-Sparse-Dense Training for Deep Neural Networks](https://mlsanity.com/paper/1607.04381/) (2016)

## Built on by (most cited first, 3 of 3)

- [Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation](https://mlsanity.com/paper/2404.10966/) (2024)
- [Unsupervised Domain Adaptation Architecture Search with Self-Training for Land Cover Mapping](https://mlsanity.com/paper/2404.14704/) (2024)
- [Awaken, Then Scale: Tiny Adaptation for MRI Reconstruction](https://mlsanity.com/paper/2609.31813/) (2026)

## Abstract (arXiv)

In this paper, we propose to develop a method to address unsupervised domain adaptation (UDA) in a practical setting of continual learning (CL). The goal is to update the model on continually changing domains while preserving domain-specific knowledge to prevent catastrophic forgetting of past-seen domains. To this end, we build a framework for preserving domain-specific features utilizing the inherent model capacity via pruning. We also perform effective inference using a novel batch-norm based metric to predict the final model parameters to be used accurately. Our approach achieves not only state-of-the-art performance but also prevents catastrophic forgetting of past domains significantly. Our code is made publicly available.