# Accurate Patient Alignment without Unnecessary Imaging Dose via Synthesizing Patient-specific 3D CT Images from 2D kV Images

arXiv: https://arxiv.org/abs/2405.19338 · HTML: https://mlsanity.com/paper/2405.19338/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)
- [PyTorch Distributed: Experiences on Accelerating Data Parallel Training](https://mlsanity.com/paper/2006.15704/) (2020)
- [X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks](https://mlsanity.com/paper/1905.06902/) (2019)
- [Deep-Learning-based Fast and Accurate 3D CT Deformable Image Registration in Lung Cancer](https://mlsanity.com/paper/2304.11135/) (2023)
- [PSA-Net: Deep Learning based Physician Style-Aware Segmentation Network for Post-Operative Prostate Cancer Clinical Target Volume](https://mlsanity.com/paper/2102.07880/) (2021)
- [CT Reconstruction from Few Planar X-rays with Application towards Low-resource Radiotherapy](https://mlsanity.com/paper/2308.02100/) (2023)
- [XTransCT: Ultra-Fast Volumetric CT Reconstruction using Two Orthogonal X-Ray Projections for Image-guided Radiation Therapy via a Transformer Network](https://mlsanity.com/paper/2305.19621/) (2023)
- [Inferring the 3D Standing Spine Posture from 2D Radiographs](https://mlsanity.com/paper/2007.06612/) (2020)
- [Deep learning-based Real-time Volumetric Imaging for Lung Stereotactic Body Radiation Therapy: A Proof of Concept Study](https://mlsanity.com/paper/2005.11832/) (2020)

## Built on by (most cited first, 1 of 1)

- [RadOnc-Agent: An LLM-Orchestrated Framework for AI Workflows Across the Radiotherapy Care Pathway](https://mlsanity.com/paper/2610.06923/) (2026)

## Abstract (arXiv)

In radiotherapy, 2D orthogonally projected kV images are used for patient alignment when 3D-on-board imaging(OBI) unavailable. But tumor visibility is constrained due to the projection of patient's anatomy onto a 2D plane, potentially leading to substantial setup errors. In treatment room with 3D-OBI such as cone beam CT(CBCT), the field of view(FOV) of CBCT is limited with unnecessarily high imaging dose, thus unfavorable for pediatric patients. A solution to this dilemma is to reconstruct 3D CT from kV images obtained at the treatment position. Here, we propose a dual-models framework built with hierarchical ViT blocks. Unlike a proof-of-concept approach, our framework considers kV images as the solo input and can synthesize accurate, full-size 3D CT in real time(within milliseconds). We demonstrate the feasibility of the proposed approach on 10 patients with head and neck (H&N) cancer using image quality(MAE: <45HU), dosimetrical accuracy(Gamma passing rate (2%/2mm/10%)>97%) and patient position uncertainty(shift error: <0.4mm). The proposed framework can generate accurate 3D CT faithfully mirroring real-time patient position, thus significantly improving patient setup accuracy, keeping imaging dose minimum, and maintaining treatment veracity.