# CoNMix for Source-free Single and Multi-target Domain Adaptation

arXiv: https://arxiv.org/abs/2211.03876 · HTML: https://mlsanity.com/paper/2211.03876/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 45 of its 61 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer](https://mlsanity.com/paper/2012.07297/) (2020)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations](https://mlsanity.com/paper/2003.12237/) (2020)
- [Gradually Vanishing Bridge for Adversarial Domain Adaptation](https://mlsanity.com/paper/2003.13183/) (2020)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)

## Built on by (most cited first, 23 of 23)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [Proxy Denoising for Source-Free Domain Adaptation](https://mlsanity.com/paper/2406.01658/) (2024)
- [Effective Restoration of Source Knowledge in Continual Test Time Adaptation](https://mlsanity.com/paper/2311.04991/) (2023)
- [Prior-guided Source-free Domain Adaptation for Human Pose Estimation](https://mlsanity.com/paper/2308.13954/) (2023)
- [Vision transformers in domain adaptation and domain generalization: a study of robustness](https://mlsanity.com/paper/2404.04452/) (2024)
- [Parameter-Selective Continual Test-Time Adaptation](https://mlsanity.com/paper/2407.02253/) (2024)
- [De-Confusing Pseudo-Labels in Source-Free Domain Adaptation](https://mlsanity.com/paper/2401.01650/) (2024)
- [Contrast and Clustering: Learning Neighborhood Pair Representation for Source-free Domain Adaptation](https://mlsanity.com/paper/2301.13428/) (2023)
- [STRIDE: Single-video based Temporally Continuous Occlusion-Robust 3D Pose Estimation](https://mlsanity.com/paper/2312.16221/) (2023)
- [Towards Practical Emotion Recognition: An Unsupervised Source-Free Approach for EEG Domain Adaptation](https://mlsanity.com/paper/2504.03707/) (2025)
- [SPA++: Generalized Graph Spectral Alignment for Versatile Domain Adaptation](https://mlsanity.com/paper/2508.05182/) (2025)
- [D3GU: Multi-Target Active Domain Adaptation via Enhancing Domain Alignment](https://mlsanity.com/paper/2401.05465/) (2024)
- [Layer-wise Model Merging for Unsupervised Domain Adaptation in Segmentation Tasks](https://mlsanity.com/paper/2409.15813/) (2024)
- [An Investigation of Test-time Adaptation for Audio Classification under Background Noise](https://mlsanity.com/paper/2507.15523/) (2025)
- [AMAuT: A Flexible and Efficient Multiview Audio Transformer Framework Trained from Scratch](https://mlsanity.com/paper/2510.19368/) (2025)
- [Exploiting Radio Frequency Fingerprints for Device Identification: Tackling Cross-receiver Challenges in the Source-data-free Scenario](https://mlsanity.com/paper/2512.16648/) (2025)
- [Source-Free Domain Adaptation with Vision-Language Prior](https://mlsanity.com/paper/2604.17748/) (2026)
- [CONTRAST: Continual Multi-source Adaptation to Dynamic Distributions](https://mlsanity.com/paper/2401.02561/) (2024)
- [Evidential Graph Contrastive Alignment for Source-Free Blending-Target Domain Adaptation](https://mlsanity.com/paper/2408.07527/) (2024)
- [DHAuDS: A Dynamic and Heterogeneous Audio Benchmark for Test-Time Adaptation](https://mlsanity.com/paper/2511.18421/) (2025)
- [Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptation](https://mlsanity.com/paper/2607.23735/) (2026)
- [Domain-Division based Progressive Learning for Source-Free Domain Adaptation](https://mlsanity.com/paper/2607.29202/) (2026)

## Abstract (arXiv)

This work introduces the novel task of Source-free Multi-target Domain Adaptation and proposes adaptation framework comprising of Consistency with Nuclear-Norm Maximization and MixUp knowledge distillation (textitCoNMix) as a solution to this problem. The main motive of this work is to solve for Single and Multi target Domain Adaptation (SMTDA) for the source-free paradigm, which enforces a constraint where the labeled source data is not available during target adaptation due to various privacy-related restrictions on data sharing. The source-free approach leverages target pseudo labels, which can be noisy, to improve the target adaptation. We introduce consistency between label preserving augmentations and utilize pseudo label refinement methods to reduce noisy pseudo labels. Further, we propose novel MixUp Knowledge Distillation (MKD) for better generalization on multiple target domains using various source-free STDA models. We also show that the Vision Transformer (VT) backbone gives better feature representation with improved domain transferability and class discriminability. Our proposed framework achieves the state-of-the-art (SOTA) results in various paradigms of source-free STDA and MTDA settings on popular domain adaptation datasets like Office-Home, Office-Caltech, and DomainNet. Project Page: https://sites.google.com/view/conmix-vcl