# Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging

arXiv: https://arxiv.org/abs/2208.12737 · HTML: https://mlsanity.com/paper/2208.12737/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 14 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 2 of 2)

- [DeepDRR -- A Catalyst for Machine Learning in Fluoroscopy-guided Procedures](https://mlsanity.com/paper/1803.08606/) (2018)
- [Predicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion](https://mlsanity.com/paper/1702.08891/) (2017)

## Built on by (most cited first, 30 of 38)

- [Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering](https://mlsanity.com/paper/2312.06358/) (2023)
- [DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering](https://mlsanity.com/paper/2406.02518/) (2024)
- [GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis](https://mlsanity.com/paper/2404.03126/) (2024)
- [RayEmb: Arbitrary Landmark Detection in X-Ray Images Using Ray Embedding Subspace](https://mlsanity.com/paper/2410.08152/) (2024)
- [An Optimization-based Baseline for Rigid 2D/3D Registration Applied to Spine Surgical Navigation Using CMA-ES](https://mlsanity.com/paper/2402.05642/) (2024)
- [Differentiable Voxel-based X-ray Rendering Improves Sparse-View 3D CBCT Reconstruction](https://mlsanity.com/paper/2411.19224/) (2024)
- [Shadow and Light: Digitally Reconstructed Radiographs for Disease Classification](https://mlsanity.com/paper/2406.03688/) (2024)
- [X-ray2CTPA: Leveraging Diffusion Models to Enhance Pulmonary Embolism Classification](https://mlsanity.com/paper/2406.16109/) (2024)
- [PolyPose: Deformable 2D/3D Registration via Polyrigid Transformations](https://mlsanity.com/paper/2505.19256/) (2025)
- [DiffUS: Differentiable Ultrasound Rendering from Volumetric Imaging](https://mlsanity.com/paper/2508.06768/) (2025)
- [Pedicle Screw Pairing and Registration for Screw Pose Estimation from Dual C-arm Images Using CAD Models](https://mlsanity.com/paper/2511.05702/) (2025)
- [X-WIN: Building Chest Radiograph World Model via Predictive Sensing](https://mlsanity.com/paper/2511.14918/) (2025)
- [AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning](https://mlsanity.com/paper/2512.17263/) (2025)
- [Active View Selection with Perturbed Gaussian Ensemble for Tomographic Reconstruction](https://mlsanity.com/paper/2603.06852/) (2026)
- [Morphology-based non-rigid registration of coronary computed tomography and intravascular images through virtual catheter path optimization](https://mlsanity.com/paper/2301.00060/) (2022)
- [Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans](https://mlsanity.com/paper/2404.11889/) (2024)
- [2D/3D Registration of Acetabular Hip Implants Under Perspective Projection and Fully Differentiable Ellipse Fitting](https://mlsanity.com/paper/2503.07763/) (2025)
- [SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model](https://mlsanity.com/paper/2507.05148/) (2025)
- [Intraoperative 2D/3D Registration via Spherical Similarity Learning and Differentiable Levenberg-Marquardt Optimization](https://mlsanity.com/paper/2509.06890/) (2025)
- [CardioComposer: Leveraging Differentiable Geometry for Compositional Control of Anatomical Diffusion Models](https://mlsanity.com/paper/2509.08015/) (2025)
- [Rapid patient-specific neural networks for intraoperative X-ray to volume registration](https://mlsanity.com/paper/2503.16309/) (2025)
- [Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints](https://mlsanity.com/paper/2506.22191/) (2025)
- [Machine-Learning Based Detection of Coronary Artery Calcification Using Synthetic Chest X-Rays](https://mlsanity.com/paper/2511.11093/) (2025)
- [Anatomica: Localized Control over Geometric and Topological Properties for Anatomical Diffusion Models](https://mlsanity.com/paper/2511.20587/) (2025)
- [Enhanced Landmark Detection Model in Pelvic Fluoroscopy using 2D/3D Registration Loss](https://mlsanity.com/paper/2511.21575/) (2025)
- [Conditional Diffusion for 3D CT Volume Reconstruction from 2D X-rays](https://mlsanity.com/paper/2603.26509/) (2026)
- [CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs](https://mlsanity.com/paper/2606.08420/) (2026)
- [Transformation-driven generation of comparable projection images from multimodal anatomical scenes](https://mlsanity.com/paper/2606.16573/) (2026)
- [From Sparse X-rays to 3D CT: Training-Free Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2606.20763/) (2026)
- [Learning Dense 2D-3D Correspondence for X-ray-to-CT Registration of Knee Bones](https://mlsanity.com/paper/2607.22803/) (2026)

## Abstract (arXiv)

The use of digitally reconstructed radiographs (DRRs) to solve inverse problems such as slice-to-volume registration and 3D reconstruction is well-studied in preoperative settings. In intraoperative imaging, the utility of DRRs is limited by the challenges in generating them in real-time and supporting optimization procedures that rely on repeated DRR synthesis. While immense progress has been made in accelerating the generation of DRRs through algorithmic refinements and GPU implementations, DRR-based optimization remains slow because most DRR generators do not offer a straightforward way to obtain gradients with respect to the imaging parameters. To make DRRs interoperable with gradient-based optimization and deep learning frameworks, we have reformulated Siddon's method, the most popular ray-tracing algorithm used in DRR generation, as a series of vectorized tensor operations. We implemented this vectorized version of Siddon's method in PyTorch, taking advantage of the library's strong automatic differentiation engine to make this DRR generator fully differentiable with respect to its parameters. Additionally, using GPU-accelerated tensor computation enables our vectorized implementation to achieve rendering speeds equivalent to state-of-the-art DRR generators implemented in CUDA and C++. We illustrate the resulting method in the context of slice-to-volume registration. Moreover, our simulations suggest that the loss landscapes for the slice-to-volume registration problem are convex in the neighborhood of the optimal solution, and gradient-based registration promises a much faster solution than prevailing gradient-free optimization strategies. The proposed DRR generator enables fast computer vision algorithms to support image guidance in minimally invasive procedures. Our implementation is publically available at https://github.com/v715/DiffDRR.