# Transfer Learning with Dynamic Distribution Adaptation

arXiv: https://arxiv.org/abs/1909.08531 · HTML: https://mlsanity.com/paper/1909.08531/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2019.

## Built on by (most cited first, 30 of 40)

- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [Deep Subdomain Adaptation Network for Image Classification](https://mlsanity.com/paper/2106.09388/) (2021)
- [Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization](https://mlsanity.com/paper/2304.14535/) (2023)
- [Single-source Domain Expansion Network for Cross-Scene Hyperspectral Image Classification](https://mlsanity.com/paper/2209.01634/) (2022)
- [Cross-domain Activity Recognition via Substructural Optimal Transport](https://mlsanity.com/paper/2102.03353/) (2021)
- [Deep transfer learning for intrusion detection in industrial control networks: A comprehensive review](https://mlsanity.com/paper/2304.10550/) (2023)
- [Language-aware Domain Generalization Network for Cross-Scene Hyperspectral Image Classification](https://mlsanity.com/paper/2209.02700/) (2022)
- [Generalizable Sleep Staging via Multi-Level Domain Alignment](https://mlsanity.com/paper/2401.05363/) (2023)
- [Domain Adaptation via Bidirectional Cross-Attention Transformer](https://mlsanity.com/paper/2201.05887/) (2022)
- [Advancing 3D Point Cloud Understanding through Deep Transfer Learning: A Comprehensive Survey](https://mlsanity.com/paper/2407.17877/) (2024)
- [FRuDA: Framework for Distributed Adversarial Domain Adaptation](https://mlsanity.com/paper/2112.13381/) (2021)
- [Explainable, Domain-Adaptive, and Federated Artificial Intelligence in Medicine](https://mlsanity.com/paper/2211.09317/) (2022)
- [EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs](https://mlsanity.com/paper/2305.00664/) (2023)
- [DAPlankton: Benchmark Dataset for Multi-instrument Plankton Recognition via Fine-grained Domain Adaptation](https://mlsanity.com/paper/2402.05615/) (2024)
- [FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment](https://mlsanity.com/paper/2501.15486/) (2025)
- [SynJAC: Synthetic-data-driven Joint-granular Adaptation and Calibration for Domain Specific Scanned Document Key Information Extraction](https://mlsanity.com/paper/2410.01609/) (2024)
- [A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms](https://mlsanity.com/paper/2109.01377/) (2021)
- [Modeling Hierarchical Structural Distance for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2211.11424/) (2022)
- [COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain Adaptation](https://mlsanity.com/paper/2308.10450/) (2023)
- [Enhancing ensemble learning and transfer learning in multimodal data analysis by adaptive dimensionality reduction](https://mlsanity.com/paper/2105.03682/) (2021)
- [An Empirical Study on Multi-Domain Robust Semantic Segmentation](https://mlsanity.com/paper/2212.04221/) (2022)
- [Self-Paced Learning for Open-Set Domain Adaptation](https://mlsanity.com/paper/2303.05933/) (2023)
- [Unlock the Potential of Counterfactually-Augmented Data in Out-Of-Distribution Generalization](https://mlsanity.com/paper/2310.06666/) (2023)
- [FedCCRL: Federated Domain Generalization with Cross-Client Representation Learning](https://mlsanity.com/paper/2410.11267/) (2024)
- [A Unified Joint Maximum Mean Discrepancy for Domain Adaptation](https://mlsanity.com/paper/2101.09979/) (2021)
- [A Framework for Supervised Heterogeneous Transfer Learning using Dynamic Distribution Adaptation and Manifold Regularization](https://mlsanity.com/paper/2108.12293/) (2021)
- [Domain Adaptation by Maximizing Population Correlation with Neural Architecture Search](https://mlsanity.com/paper/2109.06652/) (2021)

## Abstract (arXiv)

Transfer learning aims to learn robust classifiers for the target domain by leveraging knowledge from a source domain. Since the source and the target domains are usually from different distributions, existing methods mainly focus on adapting the cross-domain marginal or conditional distributions. However, in real applications, the marginal and conditional distributions usually have different contributions to the domain discrepancy. Existing methods fail to quantitatively evaluate the different importance of these two distributions, which will result in unsatisfactory transfer performance. In this paper, we propose a novel concept called Dynamic Distribution Adaptation (DDA), which is capable of quantitatively evaluating the relative importance of each distribution. DDA can be easily incorporated into the framework of structural risk minimization to solve transfer learning problems. On the basis of DDA, we propose two novel learning algorithms: (1) Manifold Dynamic Distribution Adaptation (MDDA) for traditional transfer learning, and (2) Dynamic Distribution Adaptation Network (DDAN) for deep transfer learning. Extensive experiments demonstrate that MDDA and DDAN significantly improve the transfer learning performance and setup a strong baseline over the latest deep and adversarial methods on digits recognition, sentiment analysis, and image classification. More importantly, it is shown that marginal and conditional distributions have different contributions to the domain divergence, and our DDA is able to provide good quantitative evaluation of their relative importance which leads to better performance. We believe this observation can be helpful for future research in transfer learning.