# A Two-Step Deep Learning Method for 3DCT-2DUS Kidney Registration During Breathing

arXiv: https://arxiv.org/abs/2305.13855 · HTML: https://mlsanity.com/paper/2305.13855/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 24 of 24)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registration](https://mlsanity.com/paper/1803.05982/) (2018)
- [Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks](https://mlsanity.com/paper/1907.10931/) (2019)
- [Image-and-Spatial Transformer Networks for Structure-Guided Image Registration](https://mlsanity.com/paper/1907.09200/) (2019)
- [Affine Medical Image Registration with Coarse-to-Fine Vision Transformer](https://mlsanity.com/paper/2203.15216/) (2022)
- [An Artificial Agent for Robust Image Registration](https://mlsanity.com/paper/1611.10336/) (2016)
- [Unsupervised Probabilistic Deformation Modeling for Robust Diffeomorphic Registration](https://mlsanity.com/paper/1804.07172/) (2018)
- [End-to-end Ultrasound Frame to Volume Registration](https://mlsanity.com/paper/2107.06449/) (2021)
- [Adversarial Deformation Regularization for Training Image Registration Neural Networks](https://mlsanity.com/paper/1805.10665/) (2018)
- [Multimodality Biomedical Image Registration using Free Point Transformer Networks](https://mlsanity.com/paper/2008.01885/) (2020)
- [ssEMnet: Serial-section Electron Microscopy Image Registration using a Spatial Transformer Network with Learned Features](https://mlsanity.com/paper/1707.07833/) (2017)
- [A Comparative Analysis of Registration Tools: Traditional vs Deep Learning Approach on High Resolution Tissue Cleared Data](https://mlsanity.com/paper/1810.08315/) (2018)
- [Zero Shot Learning for Multi-Modal Real Time Image Registration](https://mlsanity.com/paper/1908.06213/) (2019)
- [Attention for Image Registration (AiR): an unsupervised Transformer approach](https://mlsanity.com/paper/2105.02282/) (2021)

## Abstract (arXiv)

This work proposed a novel deep registration pipeline for 3D CT and 2D U/S kidney scans of free breathing, which consists of a feature network, and a 3D-2D CNN-based registration network. The feature network has handcraft texture feature layers to reduce the semantic gap. The registration network is encoder-decoder structure with loss of feature-image-motion (FIM), which enables hierarchical regression at decoder layers and avoids multiple network concatenation. It was first pretrained with retrospective datasets cum training data generation strategy, then adapted to specific patient data under unsupervised one-cycle transfer learning in onsite application. The experiment was on 132 U/S sequences, 39 multiple phase CT and 210 public single phase CT images, and 25 pairs of CT and U/S sequences. It resulted in mean contour distance (MCD) of 0.94 mm between kidneys on CT and U/S images and MCD of 1.15 mm on CT and reference CT images. For datasets with small transformations, it resulted in MCD of 0.82 and 1.02 mm respectively. For large transformations, it resulted in MCD of 1.10 and 1.28 mm respectively. This work addressed difficulties in 3DCT-2DUS kidney registration during free breathing via novel network structures and training strategy.