# A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy

arXiv: https://arxiv.org/abs/2004.08122 · HTML: https://mlsanity.com/paper/2004.08122/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2020.

## Built on by (most cited first, 7 of 7)

- [Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer](https://mlsanity.com/paper/2105.01844/) (2021)
- [One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs](https://mlsanity.com/paper/2201.11000/) (2022)
- [Progressively refined deep joint registration segmentation (ProRSeg) of gastrointestinal organs at risk: Application to MRI and cone-beam CT](https://mlsanity.com/paper/2210.14297/) (2022)
- [Robust One-shot Segmentation of Brain Tissues via Image-aligned Style Transformation](https://mlsanity.com/paper/2211.14521/) (2022)
- [Supervised segmentation of NO2 plumes from individual ships using TROPOMI satellite data](https://mlsanity.com/paper/2203.06993/) (2022)
- [Learning Semi-Supervised Medical Image Segmentation from Spatial Registration](https://mlsanity.com/paper/2409.10422/) (2024)
- [Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement](https://mlsanity.com/paper/2607.22770/) (2026)

## Abstract (arXiv)

Recently, joint registration and segmentation has been formulated in a deep learning setting, by the definition of joint loss functions. In this work, we investigate joining these tasks at the architectural level. We propose a registration network that integrates segmentation propagation between images, and a segmentation network to predict the segmentation directly. These networks are connected into a single joint architecture via so-called cross-stitch units, allowing information to be exchanged between the tasks in a learnable manner. The proposed method is evaluated in the context of adaptive image-guided radiotherapy, using daily prostate CT imaging. Two datasets from different institutes and manufacturers were involved in the study. The first dataset was used for training (12 patients) and validation (6 patients), while the second dataset was used as an independent test set (14 patients). In terms of mean surface distance, our approach achieved 1.06 ± 0.3 mm, 0.91 ± 0.4 mm, 1.27 ± 0.4 mm, and 1.76 ± 0.8 mm on the validation set and 1.82 ± 2.4 mm, 2.45 ± 2.4 mm, 2.45 ± 5.0 mm, and 2.57 ± 2.3 mm on the test set for the prostate, bladder, seminal vesicles, and rectum, respectively. The proposed multi-task network outperformed single-task networks, as well as a network only joined through the loss function, thus demonstrating the capability to leverage the individual strengths of the segmentation and registration tasks. The obtained performance as well as the inference speed make this a promising candidate for daily re-contouring in adaptive radiotherapy, potentially reducing treatment-related side effects and improving quality-of-life after treatment.