mlsanityData to 7 Oct 2026

RadRotator: 3D Rotation of Radiographs with Diffusion Models

arXiv 2404.13000 ↗First posted 19 Apr 2024eess.IV cs.CV cs.LGcited by 0

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Transforming two-dimensional (2D) images into three-dimensional (3D) volumes is a well-known yet challenging problem for the computer vision community. In the medical domain, a few previous studies attempted to convert two or more input radiographs into computed tomography (CT) volumes. Following their effort, we introduce a diffusion model-based technology that can rotate the anatomical content of any input radiograph in 3D space, potentially enabling the visualization of the entire anatomical content of the radiograph from any viewpoint in 3D. Similar to previous studies, we used CT volumes to create Digitally Reconstructed Radiographs (DRRs) as the training data for our model. However, we addressed two significant limitations encountered in previous studies: 1. We utilized conditional diffusion models with classifier-free guidance instead of Generative Adversarial Networks (GANs) to achieve higher mode coverage and improved output image quality, with the only trade-off being slower inference time, which is often less critical in medical applications; and 2. We demonstrated that the unreliable output of style transfer deep learning (DL) models, such as Cycle-GAN, to transfer the style of actual radiographs to DRRs could be replaced with a simple yet effective training transformation that randomly changes the pixel intensity histograms of the input and ground-truth imaging data during training. This transformation makes the diffusion model agnostic to any distribution variations of the input data pixel intensity, enabling the reliable training of a DL model on input DRRs and applying the exact same model to conventional radiographs (or DRRs) during inference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 9 of its 27 references that are in the map. First posted on arXiv 19 Apr 2024.

Builds on

9 of 27 references.

References · 9

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  7. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  8. 2019X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networkscited by 85
  9. 2018DeepDRR -- A Catalyst for Machine Learning in Fluoroscopy-guided Procedurescited by 55

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 20262D Versus 3D Diffusion for In Silico Training of Interventional X-ray AI Models
  2. 2023Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray images
  3. 2026DiffusionXRay: A Diffusion and GAN-Based Approach for Enhancing Digitally Reconstructed Chest Radiographs
  4. 2026Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation
  5. 2025Perceptual Evaluation of GANs and Diffusion Models for Generating X-rays
  6. 2025DVG-Diffusion: Dual-View Guided Diffusion Model for CT Reconstruction from X-Rays
  7. 2025SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model
  8. 2025Comparative Analysis of GAN and Diffusion for MRI-to-CT translation