# Progressively refined deep joint registration segmentation (ProRSeg) of gastrointestinal organs at risk: Application to MRI and cone-beam CT

arXiv: https://arxiv.org/abs/2210.14297 · HTML: https://mlsanity.com/paper/2210.14297/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation](https://mlsanity.com/paper/1904.08465/) (2019)
- [Self-supervised 3D anatomy segmentation using self-distilled masked image transformer (SMIT)](https://mlsanity.com/paper/2205.10342/) (2022)
- [Multi-Scale Coarse-to-Fine Segmentation for Screening Pancreatic Ductal Adenocarcinoma](https://mlsanity.com/paper/1807.02941/) (2018)
- [Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Images](https://mlsanity.com/paper/2008.00710/) (2020)
- [Recurrent Registration Neural Networks for Deformable Image Registration](https://mlsanity.com/paper/1906.09988/) (2019)
- [Fully Automated Multi-Organ Segmentation in Abdominal Magnetic Resonance Imaging with Deep Neural Networks](https://mlsanity.com/paper/1912.11000/) (2019)
- [Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration](https://mlsanity.com/paper/2104.07056/) (2021)
- [One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs](https://mlsanity.com/paper/2201.11000/) (2022)
- [A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy](https://mlsanity.com/paper/2004.08122/) (2020)

## Built on by (most cited first, 5 of 5)

- [SegReg: Segmenting OARs by Registering MR Images and CT Annotations](https://mlsanity.com/paper/2311.06956/) (2023)
- [Deep Learning for Pancreas Segmentation: a Systematic Review](https://mlsanity.com/paper/2407.16313/) (2024)
- [Modality-agnostic, patient-specific digital twins modeling temporally varying digestive motion](https://mlsanity.com/paper/2507.01909/) (2025)
- [Segmentation Regularized Training for Multi-Domain Deep Learning Registration applied to MR-Guided Prostate Cancer Radiotherapy](https://mlsanity.com/paper/2507.06966/) (2025)
- [Prediction of Rectal Cancer Regrowth from Longitudinal Endoscopy](https://mlsanity.com/paper/2605.12855/) (2026)

## Abstract (arXiv)

Method: ProRSeg was trained using 5-fold cross-validation with 110 T2-weighted MRI acquired at 5 treatment fractions from 10 different patients, taking care that same patient scans were not placed in training and testing folds. Segmentation accuracy was measured using Dice similarity coefficient (DSC) and Hausdorff distance at 95th percentile (HD95). Registration consistency was measured using coefficient of variation (CV) in displacement of OARs. Ablation tests and accuracy comparisons against multiple methods were done. Finally, applicability of ProRSeg to segment cone-beam CT (CBCT) scans was evaluated on 80 scans using 5-fold cross-validation. Results: ProRSeg processed 3D volumes (128 × 192 × 128) in 3 secs on a NVIDIA Tesla V100 GPU. It's segmentations were significantly more accurate (p<0.001) than compared methods, achieving a DSC of 0.94 ±0.02 for liver, 0.88±0.04 for large bowel, 0.78±0.03 for small bowel and 0.82±0.04 for stomach-duodenum from MRI. ProRSeg achieved a DSC of 0.72±0.01 for small bowel and 0.76±0.03 for stomach-duodenum from CBCT. ProRSeg registrations resulted in the lowest CV in displacement (stomach-duodenum CVx: 0.75%, CVy: 0.73%, and CVz: 0.81%; small bowel CVx: 0.80%, CVy: 0.80%, and CVz: 0.68%; large bowel CVx: 0.71%, CVy : 0.81%, and CVz: 0.75%). ProRSeg based dose accumulation accounting for intra-fraction (pre-treatment to post-treatment MRI scan) and inter-fraction motion showed that the organ dose constraints were violated in 4 patients for stomach-duodenum and for 3 patients for small bowel. Study limitations include lack of independent testing and ground truth phantom datasets to measure dose accumulation accuracy.