# Learning Transferable Parameters for Unsupervised Domain Adaptation

arXiv: https://arxiv.org/abs/2108.06129 · HTML: https://mlsanity.com/paper/2108.06129/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 13 Aug 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [A review of domain adaptation without target labels](https://mlsanity.com/paper/1901.05335/) (2019)
- [Asymmetric Tri-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1702.08400/) (2017)
- [Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering](https://mlsanity.com/paper/2003.08607/) (2020)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [Theoretical insights into the optimization landscape of over-parameterized shallow neural networks](https://mlsanity.com/paper/1707.04926/) (2017)
- [FINE Samples for Learning with Noisy Labels](https://mlsanity.com/paper/2102.11628/) (2021)
- [Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?](https://mlsanity.com/paper/2106.02890/) (2021)
- [Regressive Domain Adaptation for Unsupervised Keypoint Detection](https://mlsanity.com/paper/2103.06175/) (2021)

## Built on by (most cited first, 19 of 19)

- [Source-free Domain Adaptive Human Pose Estimation](https://mlsanity.com/paper/2308.03202/) (2023)
- [Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?](https://mlsanity.com/paper/2305.18712/) (2023)
- [Uncertainty-Induced Transferability Representation for Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2208.13986/) (2022)
- [StyleAM: Perception-Oriented Unsupervised Domain Adaption for Non-reference Image Quality Assessment](https://mlsanity.com/paper/2207.14489/) (2022)
- [Learning Unbiased Transferability for Domain Adaptation by Uncertainty Modeling](https://mlsanity.com/paper/2206.01319/) (2022)
- [Pseudo Label Refinery for Unsupervised Domain Adaptation on Cross-dataset 3D Object Detection](https://mlsanity.com/paper/2404.19384/) (2024)
- [Domain-Adaptive 2D Human Pose Estimation via Dual Teachers in Extremely Low-Light Conditions](https://mlsanity.com/paper/2407.15451/) (2024)
- [VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning](https://mlsanity.com/paper/2509.25033/) (2025)
- [Oracle Character Recognition using Unsupervised Discriminative Consistency Network](https://mlsanity.com/paper/2312.06075/) (2023)
- [Boosting Visual Recognition in Real-world Degradations via Unsupervised Feature Enhancement Module with Deep Channel Prior](https://mlsanity.com/paper/2404.01703/) (2024)
- [Efficient unsupervised domain adaptation via self-supervised vision transformer and synergistic cross-domain alignment](https://mlsanity.com/paper/2407.21311/) (2024)
- [Adapting Large Multimodal Models to Distribution Shifts: The Role of In-Context Learning](https://mlsanity.com/paper/2405.12217/) (2024)
- [Moses: Efficient Exploitation of Cross-device Transferable Features for Tensor Program Optimization](https://mlsanity.com/paper/2201.05752/) (2022)
- [Semantic-aware Message Broadcasting for Efficient Unsupervised Domain Adaptation](https://mlsanity.com/paper/2212.02739/) (2022)
- [UDAPose: Unsupervised Domain Adaptation for Low-Light Human Pose Estimation](https://mlsanity.com/paper/2604.10485/) (2026)
- [Generalizing to Out-of-Sample Degradations via Model Reprogramming](https://mlsanity.com/paper/2403.05886/) (2024)
- [EchoAlign: Bridging Generative and Discriminative Learning under Noisy Labels](https://mlsanity.com/paper/2405.12969/) (2024)
- [Adversarial Domain Adaptation for Cross-user Activity Recognition Using Diffusion-based Noise-centred Learning](https://mlsanity.com/paper/2408.03353/) (2024)
- [Unsupervised Attention Regularization Based Domain Adaptation for Oracle Character Recognition](https://mlsanity.com/paper/2409.15893/) (2024)

## Abstract (arXiv)

Unsupervised domain adaptation (UDA) enables a learning machine to adapt from a labeled source domain to an unlabeled domain under the distribution shift. Thanks to the strong representation ability of deep neural networks, recent remarkable achievements in UDA resort to learning domain-invariant features. Intuitively, the hope is that a good feature representation, together with the hypothesis learned from the source domain, can generalize well to the target domain. However, the learning processes of domain-invariant features and source hypothesis inevitably involve domain-specific information that would degrade the generalizability of UDA models on the target domain. In this paper, motivated by the lottery ticket hypothesis that only partial parameters are essential for generalization, we find that only partial parameters are essential for learning domain-invariant information and generalizing well in UDA. Such parameters are termed transferable parameters. In contrast, the other parameters tend to fit domain-specific details and often fail to generalize, which we term as untransferable parameters. Driven by this insight, we propose Transferable Parameter Learning (TransPar) to reduce the side effect brought by domain-specific information in the learning process and thus enhance the memorization of domain-invariant information. Specifically, according to the distribution discrepancy degree, we divide all parameters into transferable and untransferable ones in each training iteration. We then perform separate updates rules for the two types of parameters. Extensive experiments on image classification and regression tasks (keypoint detection) show that TransPar outperforms prior arts by non-trivial margins. Moreover, experiments demonstrate that TransPar can be integrated into the most popular deep UDA networks and be easily extended to handle any data distribution shift scenarios.