# Information-theoretic regularization for Multi-source Domain Adaptation

arXiv: https://arxiv.org/abs/2104.01568 · HTML: https://mlsanity.com/paper/2104.01568/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 61 references that are in the map.
- First posted on arXiv 4 Apr 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Domain-Specific Batch Normalization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1906.03950/) (2019)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [DLOW: Domain Flow for Adaptation and Generalization](https://mlsanity.com/paper/1812.05418/) (2018)
- [Image to Image Translation for Domain Adaptation](https://mlsanity.com/paper/1712.00479/) (2017)
- [Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift](https://mlsanity.com/paper/1803.00830/) (2018)
- [Adversarial Dropout Regularization](https://mlsanity.com/paper/1711.01575/) (2017)
- [Visual Domain Adaptation with Manifold Embedded Distribution Alignment](https://mlsanity.com/paper/1807.07258/) (2018)
- [Significance-aware Information Bottleneck for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/1904.00876/) (2019)
- [Multi-source Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/1910.12181/) (2019)
- [Multi-source Distilling Domain Adaptation](https://mlsanity.com/paper/1911.11554/) (2019)
- [Algorithms and Theory for Multiple-Source Adaptation](https://mlsanity.com/paper/1805.08727/) (2018)

## Built on by (most cited first, 15 of 15)

- [Balancing Discriminability and Transferability for Source-Free Domain Adaptation](https://mlsanity.com/paper/2206.08009/) (2022)
- [Subspace Identification for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2310.04723/) (2023)
- [MADG: Margin-based Adversarial Learning for Domain Generalization](https://mlsanity.com/paper/2311.08503/) (2023)
- [Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation](https://mlsanity.com/paper/2207.13247/) (2022)
- [Partial Identifiability for Domain Adaptation](https://mlsanity.com/paper/2306.06510/) (2023)
- [Multi-Source Domain Adaptation for Object Detection with Prototype-based Mean-teacher](https://mlsanity.com/paper/2309.14950/) (2023)
- [More is Better: Deep Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2405.00749/) (2024)
- [Camera Alignment and Weighted Contrastive Learning for Domain Adaptation in Video Person ReID](https://mlsanity.com/paper/2211.03626/) (2022)
- [Bridging Domains with Approximately Shared Features](https://mlsanity.com/paper/2403.06424/) (2024)
- [A General Representation-Based Approach to Multi-Source Domain Adaptation](https://mlsanity.com/paper/2604.23790/) (2026)
- [Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI](https://mlsanity.com/paper/2604.21300/) (2026)
- [Maximizing Information in Domain-Invariant Representation Improves Transfer Learning](https://mlsanity.com/paper/2306.00262/) (2023)
- [AED-PADA:Improving Generalizability of Adversarial Example Detection via Principal Adversarial Domain Adaptation](https://mlsanity.com/paper/2404.12635/) (2024)
- [Adaptive Hardness-driven Augmentation and Alignment Strategies for Multi-Source Domain Adaptations](https://mlsanity.com/paper/2501.01142/) (2025)
- [Towards Conditional Feature Alignment for Cross-Domain Counting](https://mlsanity.com/paper/2506.17137/) (2025)

## Abstract (arXiv)

Adversarial learning strategy has demonstrated remarkable performance in dealing with single-source Domain Adaptation (DA) problems, and it has recently been applied to Multi-source DA (MDA) problems. Although most existing MDA strategies rely on a multiple domain discriminator setting, its effect on the latent space representations has been poorly understood. Here we adopt an information-theoretic approach to identify and resolve the potential adverse effect of the multiple domain discriminators on MDA: disintegration of domain-discriminative information, limited computational scalability, and a large variance in the gradient of the loss during training. We examine the above issues by situating adversarial DA in the context of information regularization. This also provides a theoretical justification for using a single and unified domain discriminator. Based on this idea, we implement a novel neural architecture called a Multi-source Information-regularized Adaptation Networks (MIAN). Large-scale experiments demonstrate that MIAN, despite its structural simplicity, reliably and significantly outperforms other state-of-the-art methods.