# Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

arXiv: https://arxiv.org/abs/2103.03781 · HTML: https://mlsanity.com/paper/2103.03781/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 56 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation](https://mlsanity.com/paper/1907.10830/) (2019)
- [Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation](https://mlsanity.com/paper/2002.02255/) (2020)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [Medical Image Synthesis with Context-Aware Generative Adversarial Networks](https://mlsanity.com/paper/1612.05362/) (2016)
- [Deep MR to CT Synthesis using Unpaired Data](https://mlsanity.com/paper/1708.01155/) (2017)
- [Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis](https://mlsanity.com/paper/2002.05000/) (2020)
- [Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation](https://mlsanity.com/paper/2006.16806/) (2020)
- [Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis](https://mlsanity.com/paper/2007.07222/) (2020)
- [Unsupervised Attention-guided Image to Image Translation](https://mlsanity.com/paper/1806.02311/) (2018)
- [Label Efficient Learning of Transferable Representations across Domains and Tasks](https://mlsanity.com/paper/1712.00123/) (2017)
- [Deep CT to MR Synthesis using Paired and Unpaired Data](https://mlsanity.com/paper/1805.10790/) (2018)
- [Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs)](https://mlsanity.com/paper/1707.09747/) (2017)
- [SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis](https://mlsanity.com/paper/1908.04346/) (2019)
- [PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network with a Benchmark at Cross-modality Cardiac Segmentation](https://mlsanity.com/paper/1812.07907/) (2018)
- [Progressive Generative Adversarial Networks for Medical Image Super resolution](https://mlsanity.com/paper/1902.02144/) (2019)

## Built on by (most cited first, 24 of 24)

- [Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives](https://mlsanity.com/paper/2308.01265/) (2023)
- [FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation](https://mlsanity.com/paper/2304.13672/) (2023)
- [Multi-Modality Cardiac Image Computing: A Survey](https://mlsanity.com/paper/2208.12881/) (2022)
- [Prototypical few-shot segmentation for cross-institution male pelvic structures with spatial registration](https://mlsanity.com/paper/2209.05160/) (2022)
- [Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images](https://mlsanity.com/paper/2103.05745/) (2021)
- [Unsupervised Model Adaptation for Source-free Segmentation of Medical Images](https://mlsanity.com/paper/2211.00807/) (2022)
- [Is attention all you need in medical image analysis? A review](https://mlsanity.com/paper/2307.12775/) (2023)
- [Self-Supervised Generative Style Transfer for One-Shot Medical Image Segmentation](https://mlsanity.com/paper/2110.02117/) (2021)
- [SDC-UDA: Volumetric Unsupervised Domain Adaptation Framework for Slice-Direction Continuous Cross-Modality Medical Image Segmentation](https://mlsanity.com/paper/2305.11012/) (2023)
- [Cross-Modality Neuroimage Synthesis: A Survey](https://mlsanity.com/paper/2202.06997/) (2022)
- [ASC: Appearance and Structure Consistency for Unsupervised Domain Adaptation in Fetal Brain MRI Segmentation](https://mlsanity.com/paper/2310.14172/) (2023)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [Diffuse-UDA: Addressing Unsupervised Domain Adaptation in Medical Image Segmentation with Appearance and Structure Aligned Diffusion Models](https://mlsanity.com/paper/2408.05985/) (2024)
- [Medical Image Segmentation with Domain Adaptation: A Survey](https://mlsanity.com/paper/2311.01702/) (2023)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [An Attentive-based Generative Model for Medical Image Synthesis](https://mlsanity.com/paper/2306.01562/) (2023)
- [Cross-Domain Distribution Alignment for Segmentation of Private Unannotated 3D Medical Images](https://mlsanity.com/paper/2410.09210/) (2024)
- [Fully Differentiable Bidirectional Dual-Task Synergistic Learning for Semi-Supervised 3D Medical Image Segmentation](https://mlsanity.com/paper/2602.09378/) (2026)
- [Contrastive Image Synthesis and Self-supervised Feature Adaptation for Cross-Modality Biomedical Image Segmentation](https://mlsanity.com/paper/2207.13240/) (2022)
- [SL: Stable Learning in Source-Free Domain Adaption for Medical Image Segmentation](https://mlsanity.com/paper/2307.12580/) (2023)
- [Gaze-Assisted Human-Centric Domain Adaptation for Cardiac Ultrasound Image Segmentation](https://mlsanity.com/paper/2502.03781/) (2025)
- [Learning Semantic Directions for Feature Augmentation in Domain-Generalized Medical Segmentation](https://mlsanity.com/paper/2507.23326/) (2025)
- [Unsupervised Domain Adaptation via Similarity-based Prototypes for Cross-Modality Segmentation](https://mlsanity.com/paper/2510.20596/) (2025)
- [SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation](https://mlsanity.com/paper/2603.21904/) (2026)

## Abstract (arXiv)

Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solution is based on the cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists and bridge the domain gap in radiological images. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups. Built upon adversarial training, we propose a learnable self-attentive spatial normalization of the deep convolutional generator network's intermediate activations. Unlike previous attention-based image-to-image translation approaches, which are either domain-specific or require distortion of the source domain's structures, we unearth the importance of the auxiliary semantic information to handle the geometric changes and preserve anatomical structures during image translation. We achieve superior results for cross-modality segmentation between unpaired MRI and CT data for multi-modality whole heart and multi-modal brain tumor MRI (T1/T2) datasets compared to the state-of-the-art methods. We also observe encouraging results in cross-modality conversion for paired MRI and CT images on a brain dataset. Furthermore, a detailed analysis of the cross-modality image translation, thorough ablation studies confirm our proposed method's efficacy.