# DeepDRR -- A Catalyst for Machine Learning in Fluoroscopy-guided Procedures

arXiv: https://arxiv.org/abs/1803.08606 · HTML: https://mlsanity.com/paper/1803.08606/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2018.

## Built on by (most cited first, 30 of 55)

- [Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging](https://mlsanity.com/paper/2208.12737/) (2022)
- [Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering](https://mlsanity.com/paper/2312.06358/) (2023)
- [DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering](https://mlsanity.com/paper/2406.02518/) (2024)
- [The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspective](https://mlsanity.com/paper/2108.02238/) (2021)
- [Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future](https://mlsanity.com/paper/2108.04543/) (2021)
- [Self-Supervised 2D/3D Registration for X-Ray to CT Image Fusion](https://mlsanity.com/paper/2210.07611/) (2022)
- [Pelphix: Surgical Phase Recognition from X-ray Images in Percutaneous Pelvic Fixation](https://mlsanity.com/paper/2304.09285/) (2023)
- [FluoroSAM: A Language-promptable Foundation Model for Flexible X-ray Image Segmentation](https://mlsanity.com/paper/2403.08059/) (2024)
- [Synthetic Data in Radiological Imaging: Current State and Future Outlook](https://mlsanity.com/paper/2407.01561/) (2024)
- [Denoising Diffusion Medical Models](https://mlsanity.com/paper/2304.09383/) (2023)
- [SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments](https://mlsanity.com/paper/2206.06127/) (2022)
- [PELE scores: Pelvic X-ray Landmark Detection by Pelvis Extraction and Enhancement](https://mlsanity.com/paper/2305.04294/) (2023)
- [Benchmark of Segmentation Techniques for Pelvic Fracture in CT and X-ray: Summary of the PENGWIN 2024 Challenge](https://mlsanity.com/paper/2504.02382/) (2025)
- [Transfer Learning from an Artificial Radiograph-landmark Dataset for Registration of the Anatomic Skull Model to Dual Fluoroscopic X-ray Images](https://mlsanity.com/paper/2108.06466/) (2021)
- [View-Consistent Metal Segmentation in the Projection Domain for Metal Artifact Reduction in CBCT -- An Investigation of Potential Improvement](https://mlsanity.com/paper/2112.02101/) (2021)
- [Neural Radiance Projection](https://mlsanity.com/paper/2203.07658/) (2022)
- [Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data](https://mlsanity.com/paper/2401.16027/) (2024)
- [Physics-Informed Learning for Time-Resolved Angiographic Contrast Agent Concentration Reconstruction](https://mlsanity.com/paper/2403.01993/) (2024)
- [Pedicle Screw Pairing and Registration for Screw Pose Estimation from Dual C-arm Images Using CAD Models](https://mlsanity.com/paper/2511.05702/) (2025)
- [A Frequency Domain Constraint for Synthetic and Real X-ray Image Super Resolution](https://mlsanity.com/paper/2105.06887/) (2021)
- [CNN-based real-time 2D-3D deformable registration from a single X-ray projection](https://mlsanity.com/paper/2212.07692/) (2022)
- [Shape-based pose estimation for automatic standard views of the knee](https://mlsanity.com/paper/2305.16717/) (2023)
- [Segment Any Medical Model Extended](https://mlsanity.com/paper/2403.18114/) (2024)
- [Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans](https://mlsanity.com/paper/2404.11889/) (2024)
- [2D/3D Registration of Acetabular Hip Implants Under Perspective Projection and Fully Differentiable Ellipse Fitting](https://mlsanity.com/paper/2503.07763/) (2025)
- [Self-Supervised Multiview Xray Matching](https://mlsanity.com/paper/2507.00287/) (2025)
- [Human-AI Collaboration and Explainability for 2D/3D Registration Quality Assurance](https://mlsanity.com/paper/2507.17597/) (2025)
- [Intraoperative 2D/3D Registration via Spherical Similarity Learning and Differentiable Levenberg-Marquardt Optimization](https://mlsanity.com/paper/2509.06890/) (2025)
- [C-arm Guidance: A Self-supervised Approach To Automated Positioning During Stroke Thrombectomy](https://mlsanity.com/paper/2510.16145/) (2025)
- [Automated C-Arm Positioning via Conformal Landmark Localization](https://mlsanity.com/paper/2510.16160/) (2025)

## Abstract (arXiv)

Machine learning-based approaches outperform competing methods in most disciplines relevant to diagnostic radiology. Interventional radiology, however, has not yet benefited substantially from the advent of deep learning, in particular because of two reasons: 1) Most images acquired during the procedure are never archived and are thus not available for learning, and 2) even if they were available, annotations would be a severe challenge due to the vast amounts of data. When considering fluoroscopy-guided procedures, an interesting alternative to true interventional fluoroscopy is in silico simulation of the procedure from 3D diagnostic CT. In this case, labeling is comparably easy and potentially readily available, yet, the appropriateness of resulting synthetic data is dependent on the forward model. In this work, we propose DeepDRR, a framework for fast and realistic simulation of fluoroscopy and digital radiography from CT scans, tightly integrated with the software platforms native to deep learning. We use machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, combined with analytic forward projection and noise injection to achieve the required performance. On the example of anatomical landmark detection in X-ray images of the pelvis, we demonstrate that machine learning models trained on DeepDRRs generalize to unseen clinically acquired data without the need for re-training or domain adaptation. Our results are promising and promote the establishment of machine learning in fluoroscopy-guided procedures.