# Attention-based Cross-Layer Domain Alignment for Unsupervised Domain Adaptation

arXiv: https://arxiv.org/abs/2202.13310 · HTML: https://mlsanity.com/paper/2202.13310/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 27 Feb 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Asymmetric Tri-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1702.08400/) (2017)
- [Cross-Layer Distillation with Semantic Calibration](https://mlsanity.com/paper/2012.03236/) (2020)
- [Transferrable Prototypical Networks for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1904.11227/) (2019)
- [Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering](https://mlsanity.com/paper/2003.08607/) (2020)
- [Gradually Vanishing Bridge for Adversarial Domain Adaptation](https://mlsanity.com/paper/2003.13183/) (2020)
- [Domain-Symmetric Networks for Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.04663/) (2019)
- [Cluster Alignment with a Teacher for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1903.09980/) (2019)
- [Adversarial-Learned Loss for Domain Adaptation](https://mlsanity.com/paper/2001.01046/) (2020)
- [Spatial Attention Pyramid Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2003.12979/) (2020)
- [Domain-Specific Bias Filtering for Single Labeled Domain Generalization](https://mlsanity.com/paper/2110.00726/) (2021)
- [Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization](https://mlsanity.com/paper/2110.06736/) (2021)
- [Cycle-consistent Conditional Adversarial Transfer Networks](https://mlsanity.com/paper/1909.07618/) (2019)
- [Unsupervised Domain Adaptation via Discriminative Manifold Embedding and Alignment](https://mlsanity.com/paper/2002.08675/) (2020)

## Built on by (most cited first, 4 of 4)

- [Label-Efficient Domain Generalization via Collaborative Exploration and Generalization](https://mlsanity.com/paper/2208.03644/) (2022)
- [Universal Domain Adaptation via Compressive Attention Matching](https://mlsanity.com/paper/2304.11862/) (2023)
- [Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on Medical Image Segmentation](https://mlsanity.com/paper/2211.15235/) (2022)
- [Generalized Universal Domain Adaptation with Generative Flow Networks](https://mlsanity.com/paper/2305.04466/) (2023)

## Abstract (arXiv)

Unsupervised domain adaptation (UDA) aims to learn transferable knowledge from a labeled source domain and adapts a trained model to an unlabeled target domain. To bridge the gap between source and target domains, one prevailing strategy is to minimize the distribution discrepancy by aligning their semantic features extracted by deep models. The existing alignment-based methods mainly focus on reducing domain divergence in the same model layer. However, the same level of semantic information could distribute across model layers due to the domain shifts. To further boost model adaptation performance, we propose a novel method called Attention-based Cross-layer Domain Alignment (ACDA), which captures the semantic relationship between the source and target domains across model layers and calibrates each level of semantic information automatically through a dynamic attention mechanism. An elaborate attention mechanism is designed to reweight each cross-layer pair based on their semantic similarity for precise domain alignment, effectively matching each level of semantic information during model adaptation. Extensive experiments on multiple benchmark datasets consistently show that the proposed method ACDA yields state-of-the-art performance.