# Revisiting Cross-Attention Mechanisms: Leveraging Beneficial Noise for Domain-Adaptive Learning

arXiv: https://arxiv.org/abs/2603.17474 · HTML: https://mlsanity.com/paper/2603.17474/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [StyTr2: Image Style Transfer with Transformers](https://mlsanity.com/paper/2105.14576/) (2021)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)
- [FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2011.09230/) (2020)
- [Rewriting a Deep Generative Model](https://mlsanity.com/paper/2007.15646/) (2020)
- [Fast Patch-based Style Transfer of Arbitrary Style](https://mlsanity.com/paper/1612.04337/) (2016)
- [TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2108.05988/) (2021)
- [Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation](https://mlsanity.com/paper/2104.09415/) (2021)
- [Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.04151/) (2021)
- [Adversarial-Learned Loss for Domain Adaptation](https://mlsanity.com/paper/2001.01046/) (2020)
- [Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective](https://mlsanity.com/paper/2303.13434/) (2023)
- [Fast-iTPN: Integrally Pre-Trained Transformer Pyramid Network with Token Migration](https://mlsanity.com/paper/2211.12735/) (2022)
- [Boosting Novel Category Discovery Over Domains with Soft Contrastive Learning and All-in-One Classifier](https://mlsanity.com/paper/2211.11262/) (2022)
- [Domain Adaptation via Bidirectional Cross-Attention Transformer](https://mlsanity.com/paper/2201.05887/) (2022)
- [DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment](https://mlsanity.com/paper/2405.11765/) (2024)
- [Making the Best of Both Worlds: A Domain-Oriented Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2208.01195/) (2022)
- [Aligning Non-Causal Factors for Transformer-Based Source-Free Domain Adaptation](https://mlsanity.com/paper/2311.16294/) (2023)

## Abstract (arXiv)

Unsupervised Domain Adaptation (UDA) seeks to transfer knowledge from a labeled source domain to an unlabeled target domain but often suffers from severe domain and scale gaps that degrade performance. Existing cross-attention-based transformers can align features across domains, yet they struggle to preserve content semantics under large appearance and scale variations. To explicitly address these challenges, we introduce the concept of beneficial noise, which regularizes cross-attention by injecting controlled perturbations, encouraging the model to ignore style distractions and focus on content. We propose the Domain-Adaptive Cross-Scale Matching (DACSM) framework, which consists of a Domain-Adaptive Transformer (DAT) for disentangling domain-shared content from domain-specific style, and a Cross-Scale Matching (CSM) module that adaptively aligns features across multiple resolutions. DAT incorporates beneficial noise into cross-attention, enabling progressive domain translation with enhanced robustness, yielding content-consistent and style-invariant representations. Meanwhile, CSM ensures semantic consistency under scale changes. Extensive experiments on VisDA-2017, Office-Home, and DomainNet demonstrate that DACSM achieves state-of-the-art performance, with up to +2.3% improvement over CDTrans on VisDA-2017. Notably, DACSM achieves a +5.9% gain on the challenging "truck" class of VisDA, evidencing the strength of beneficial noise in handling scale discrepancies. These results highlight the effectiveness of combining domain translation, beneficial-noise-enhanced attention, and scale-aware alignment for robust cross-domain representation learning.