# Synthetic CT Generation from MRI using 3D Transformer-based Denoising Diffusion Model

arXiv: https://arxiv.org/abs/2305.19467 · HTML: https://mlsanity.com/paper/2305.19467/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 31 May 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2001.06937/) (2020)
- [On Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.00132/) (2021)
- [Deep MR to CT Synthesis using Unpaired Data](https://mlsanity.com/paper/1708.01155/) (2017)
- [Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models](https://mlsanity.com/paper/2209.12104/) (2022)
- [Unpaired Brain MR-to-CT Synthesis using a Structure-Constrained CycleGAN](https://mlsanity.com/paper/1809.04536/) (2018)
- [Cycle-guided Denoising Diffusion Probability Model for 3D Cross-modality MRI Synthesis](https://mlsanity.com/paper/2305.00042/) (2023)
- [Cross-Shaped Windows Transformer with Self-supervised Pretraining for Clinically Significant Prostate Cancer Detection in Bi-parametric MRI](https://mlsanity.com/paper/2305.00385/) (2023)

## Built on by (most cited first, 29 of 29)

- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges](https://mlsanity.com/paper/2407.00116/) (2024)
- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [Diffusion-empowered AutoPrompt MedSAM](https://mlsanity.com/paper/2502.06817/) (2025)
- [Cascaded Multi-path Shortcut Diffusion Model for Medical Image Translation](https://mlsanity.com/paper/2405.12223/) (2024)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [Benchmarking GANs, Diffusion Models, and Flow Matching for T1w-to-T2w MRI Translation](https://mlsanity.com/paper/2507.14575/) (2025)
- [3D Wavelet Latent Diffusion Model for Whole-Body MR-to-CT Modality Translation](https://mlsanity.com/paper/2507.11557/) (2025)
- [Artificial Intelligence for Biomedical Video Generation](https://mlsanity.com/paper/2411.07619/) (2024)
- [MRI-to-CT synthesis using drifting models](https://mlsanity.com/paper/2603.28498/) (2026)
- [Unsupervised Skull Segmentation via Contrastive MR-to-CT Modality Translation](https://mlsanity.com/paper/2410.13427/) (2024)
- [Structure-Guided MR-to-CT Synthesis with Spatial and Semantic Alignments for Attenuation Correction of Whole-Body PET/MR Imaging](https://mlsanity.com/paper/2411.17488/) (2024)
- [Cascaded 3D Diffusion Models for Whole-body 3D 18-F FDG PET/CT synthesis from Demographics](https://mlsanity.com/paper/2505.22489/) (2025)
- [Surf2CT: Cascaded 3D Flow Matching Models for Torso 3D CT Synthesis from Skin Surface](https://mlsanity.com/paper/2505.22511/) (2025)
- [A Unified Framework for Multimodal Image Reconstruction and Synthesis using Denoising Diffusion Models](https://mlsanity.com/paper/2602.08249/) (2026)
- [Cross Modality Image Translation In Medical Imaging Using Generative Frameworks](https://mlsanity.com/paper/2605.13686/) (2026)
- [Knowledge-Guided 3D CT Generation: A Conditioning-Centric Taxonomy](https://mlsanity.com/paper/2608.09992/) (2026)
- [Attention-Driven Lightweight Model for Pigmented Skin Lesion Detection](https://mlsanity.com/paper/2308.02119/) (2023)
- [Advancing MRI Reconstruction: A Systematic Review of Deep Learning and Compressed Sensing Integration](https://mlsanity.com/paper/2501.14158/) (2025)
- [Path and Bone-Contour Regularized Unpaired MRI-to-CT Translation](https://mlsanity.com/paper/2505.03114/) (2025)
- [GM-LDM: Latent Diffusion Model for Brain Biomarker Identification through Functional Data-Driven Gray Matter Synthesis](https://mlsanity.com/paper/2506.12719/) (2025)
- [Generalizable 7T T1-map Synthesis from 1.5T and 3T T1 MRI with an Efficient Transformer Model](https://mlsanity.com/paper/2507.08655/) (2025)
- [MInDI-3D: Iterative Deep Learning in 3D for Sparse-view Cone Beam Computed Tomography](https://mlsanity.com/paper/2508.09616/) (2025)
- [An Efficient 3D Latent Diffusion Model for T1-contrast Enhanced MRI Generation](https://mlsanity.com/paper/2509.24194/) (2025)
- [Improving Diagnostic Performance on Small and Imbalanced Datasets Using Class-Based Input Image Composition](https://mlsanity.com/paper/2511.03891/) (2025)
- [VolDiT: Controllable Volumetric Medical Image Synthesis with Diffusion Transformers](https://mlsanity.com/paper/2603.25181/) (2026)
- [VS-DDPM: Efficient Low-Cost Diffusion Model for Medical Modality Translation](https://mlsanity.com/paper/2604.22942/) (2026)
- [A Proof-of-Concept Study of Multitask Learning for Cranial Synthetic CT Generation Across Heterogeneous MRI Field Strengths](https://mlsanity.com/paper/2605.00923/) (2026)
- [MRecover: A Conditional Generative Model for Recovering Motion-Corrupted MR images Using AI Generated Contrast](https://mlsanity.com/paper/2605.21669/) (2026)

## Abstract (arXiv)

Magnetic resonance imaging (MRI)-based synthetic computed tomography (sCT) simplifies radiation therapy treatment planning by eliminating the need for CT simulation and error-prone image registration, ultimately reducing patient radiation dose and setup uncertainty. We propose an MRI-to-CT transformer-based denoising diffusion probabilistic model (MC-DDPM) to transform MRI into high-quality sCT to facilitate radiation treatment planning. MC-DDPM implements diffusion processes with a shifted-window transformer network to generate sCT from MRI. The proposed model consists of two processes: a forward process which adds Gaussian noise to real CT scans, and a reverse process in which a shifted-window transformer V-net (Swin-Vnet) denoises the noisy CT scans conditioned on the MRI from the same patient to produce noise-free CT scans. With an optimally trained Swin-Vnet, the reverse diffusion process was used to generate sCT scans matching MRI anatomy. We evaluated the proposed method by generating sCT from MRI on a brain dataset and a prostate dataset. Qualitative evaluation was performed using the mean absolute error (MAE) of Hounsfield unit (HU), peak signal to noise ratio (PSNR), multi-scale Structure Similarity index (MS-SSIM) and normalized cross correlation (NCC) indexes between ground truth CTs and sCTs. MC-DDPM generated brain sCTs with state-of-the-art quantitative results with MAE 43.317 HU, PSNR 27.046 dB, SSIM 0.965, and NCC 0.983. For the prostate dataset, MC-DDPM achieved MAE 59.953 HU, PSNR 26.920 dB, SSIM 0.849, and NCC 0.948. In conclusion, we have developed and validated a novel approach for generating CT images from routine MRIs using a transformer-based DDPM. This model effectively captures the complex relationship between CT and MRI images, allowing for robust and high-quality synthetic CT (sCT) images to be generated in minutes.