# RT-SRTS: Angle-Agnostic Real-Time Simultaneous 3D Reconstruction and Tumor Segmentation from Single X-Ray Projection

arXiv: https://arxiv.org/abs/2310.08080 · HTML: https://mlsanity.com/paper/2310.08080/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 54 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks](https://mlsanity.com/paper/1905.06902/) (2019)
- [Multi-phase Liver Tumor Segmentation with Spatial Aggregation and Uncertain Region Inpainting](https://mlsanity.com/paper/2108.00911/) (2021)
- [Deep Unfolding of the DBFB Algorithm with Application to ROI CT Imaging with Limited Angular Density](https://mlsanity.com/paper/2209.13264/) (2022)
- [Deep learning-based Real-time Volumetric Imaging for Lung Stereotactic Body Radiation Therapy: A Proof of Concept Study](https://mlsanity.com/paper/2005.11832/) (2020)

## Abstract (arXiv)

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.