# Cross-Modality Multi-Atlas Segmentation via Deep Registration and Label Fusion

arXiv: https://arxiv.org/abs/2202.02000 · HTML: https://mlsanity.com/paper/2202.02000/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation](https://mlsanity.com/paper/2002.02255/) (2020)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [Multi-Atlas Segmentation of Biomedical Images: A Survey](https://mlsanity.com/paper/1412.3421/) (2014)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation](https://mlsanity.com/paper/1904.08963/) (2019)
- [Cardiac Segmentation on Late Gadolinium Enhancement MRI: A Benchmark Study from Multi-Sequence Cardiac MR Segmentation Challenge](https://mlsanity.com/paper/2006.12434/) (2020)
- [BIRNet: Brain Image Registration Using Dual-Supervised Fully Convolutional Networks](https://mlsanity.com/paper/1802.04692/) (2018)
- [Unsupervised Deformable Image Registration Using Cycle-Consistent CNN](https://mlsanity.com/paper/1907.01319/) (2019)
- [Atrial Scar Quantification via Multi-scale CNN in the Graph-cuts Framework](https://mlsanity.com/paper/1902.07877/) (2019)
- [MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation](https://mlsanity.com/paper/2006.15573/) (2020)
- [Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images](https://mlsanity.com/paper/1709.09641/) (2017)
- [Cross-Modality Multi-Atlas Segmentation Using Deep Neural Networks](https://mlsanity.com/paper/2008.08946/) (2020)

## Built on by (most cited first, 8 of 8)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Multi-Modality Cardiac Image Computing: A Survey](https://mlsanity.com/paper/2208.12881/) (2022)
- [DiffAtlas: GenAI-fying Atlas Segmentation via Image-Mask Diffusion](https://mlsanity.com/paper/2503.06748/) (2025)
- [AVP-AP: Self-supervised Automatic View Positioning in 3D cardiac CT via Atlas Prompting](https://mlsanity.com/paper/2504.05966/) (2025)
- [Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey](https://mlsanity.com/paper/2507.22791/) (2025)
- [GenMed: A Pairwise Generative Reformulation of Medical Diagnostic Tasks](https://mlsanity.com/paper/2605.10645/) (2026)
- [Deep learning in medical image registration: introduction and survey](https://mlsanity.com/paper/2309.00727/) (2023)
- [Ultrasound Image Segmentation of Thyroid Nodule via Latent Semantic Feature Co-Registration](https://mlsanity.com/paper/2310.09221/) (2023)

## Abstract (arXiv)

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image; and the transformed atlas labels can be combined to generate target segmentation via label fusion schemes. Many conventional MAS methods employed the atlases from the same modality as the target image. However, the number of atlases with the same modality may be limited or even missing in many clinical applications. Besides, conventional MAS methods suffer from the computational burden of registration or label fusion procedures. In this work, we design a novel cross-modality MAS framework, which uses available atlases from a certain modality to segment a target image from another modality. To boost the computational efficiency of the framework, both the image registration and label fusion are achieved by well-designed deep neural networks. For the atlas-to-target image registration, we propose a bi-directional registration network (BiRegNet), which can efficiently align images from different modalities. For the label fusion, we design a similarity estimation network (SimNet), which estimates the fusion weight of each atlas by measuring its similarity to the target image. SimNet can learn multi-scale information for similarity estimation to improve the performance of label fusion. The proposed framework was evaluated by the left ventricle and liver segmentation tasks on the MM-WHS and CHAOS datasets, respectively. Results have shown that the framework is effective for cross-modality MAS in both registration and label fusion.