# Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks

arXiv: https://arxiv.org/abs/2006.16741 · HTML: https://mlsanity.com/paper/2006.16741/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2020.

## Built on by (most cited first, 10 of 10)

- [Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives](https://mlsanity.com/paper/2308.01265/) (2023)
- [Transfer Learning in Magnetic Resonance Brain Imaging: a Systematic Review](https://mlsanity.com/paper/2102.01530/) (2021)
- [Embracing the Disharmony in Medical Imaging: A Simple and Effective Framework for Domain Adaptation](https://mlsanity.com/paper/2103.12857/) (2021)
- [Harmonization with Flow-based Causal Inference](https://mlsanity.com/paper/2106.06845/) (2021)
- [Segmentation-guided Domain Adaptation and Data Harmonization of Multi-device Retinal Optical Coherence Tomography using Cycle-Consistent Generative Adversarial Networks](https://mlsanity.com/paper/2208.14635/) (2022)
- [Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data](https://mlsanity.com/paper/2211.02400/) (2022)
- [Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images](https://mlsanity.com/paper/2104.09699/) (2021)
- [IGUANe: a 3D generalizable CycleGAN for multicenter harmonization of brain MR images](https://mlsanity.com/paper/2402.03227/) (2024)
- [Meta-hallucinator: Towards Few-Shot Cross-Modality Cardiac Image Segmentation](https://mlsanity.com/paper/2305.06978/) (2023)
- [Adaptation of Tongue Ultrasound-Based Silent Speech Interfaces Using Spatial Transformer Networks](https://mlsanity.com/paper/2305.19130/) (2023)

## Abstract (arXiv)

We investigate the use of image-and-spatial transformer networks (ISTNs) to tackle domain shift in multi-site medical imaging data. Commonly, domain adaptation (DA) is performed with little regard for explainability of the inter-domain transformation and is often conducted at the feature-level in the latent space. We employ ISTNs for DA at the image-level which constrains transformations to explainable appearance and shape changes. As proof-of-concept we demonstrate that ISTNs can be trained adversarially on a classification problem with simulated 2D data. For real-data validation, we construct two 3D brain MRI datasets from the Cam-CAN and UK Biobank studies to investigate domain shift due to acquisition and population differences. We show that age regression and sex classification models trained on ISTN output improve generalization when training on data from one and testing on the other site.