# SepRep-Net: Multi-source Free Domain Adaptation via Model Separation And Reparameterization

arXiv: https://arxiv.org/abs/2402.08249 · HTML: https://mlsanity.com/paper/2402.08249/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 65 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021)
- [Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1903.04064/) (2019)
- [Explicit Inductive Bias for Transfer Learning with Convolutional Networks](https://mlsanity.com/paper/1802.01483/) (2018)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [Multi-Adversarial Domain Adaptation](https://mlsanity.com/paper/1809.02176/) (2018)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer](https://mlsanity.com/paper/2012.07297/) (2020)
- [Universal Domain Adaptation through Self Supervision](https://mlsanity.com/paper/2002.07953/) (2020)
- [Transferrable Prototypical Networks for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1904.11227/) (2019)
- [Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering](https://mlsanity.com/paper/2003.08607/) (2020)
- [Domain-Symmetric Networks for Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.04663/) (2019)
- [Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift](https://mlsanity.com/paper/1803.00830/) (2018)

## Built on by (most cited first, 2 of 2)

- [Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training](https://mlsanity.com/paper/2405.02954/) (2024)
- [Selection, Ensemble, and Adaptation: Advancing Multi-Source-Free Domain Adaptation via Architecture Zoo](https://mlsanity.com/paper/2403.01582/) (2024)

## Abstract (arXiv)

We consider multi-source free domain adaptation, the problem of adapting multiple existing models to a new domain without accessing the source data. Among existing approaches, methods based on model ensemble are effective in both the source and target domains, but incur significantly increased computational costs. Towards this dilemma, in this work, we propose a novel framework called SepRep-Net, which tackles multi-source free domain adaptation via model Separation and Reparameterization.Concretely, SepRep-Net reassembled multiple existing models to a unified network, while maintaining separate pathways (Separation). During training, separate pathways are optimized in parallel with the information exchange regularly performed via an additional feature merging unit. With our specific design, these pathways can be further reparameterized into a single one to facilitate inference (Reparameterization). SepRep-Net is characterized by 1) effectiveness: competitive performance on the target domain, 2) efficiency: low computational costs, and 3) generalizability: maintaining more source knowledge than existing solutions. As a general approach, SepRep-Net can be seamlessly plugged into various methods. Extensive experiments validate the performance of SepRep-Net on mainstream benchmarks.