# Patient-Agnostic Synthetic Pretraining for Efficient Patient-Specific Intraoperative 2D/3D Registration

arXiv: https://arxiv.org/abs/2607.23343 · HTML: https://mlsanity.com/paper/2607.23343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 73 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 30 of 39)

- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation](https://mlsanity.com/paper/2309.08842/) (2023)
- [DeepDRR -- A Catalyst for Machine Learning in Fluoroscopy-guided Procedures](https://mlsanity.com/paper/1803.08606/) (2018)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets](https://mlsanity.com/paper/2209.02032/) (2022)
- [Beyond Adapting SAM: Towards End-to-End Ultrasound Image Segmentation via Auto Prompting](https://mlsanity.com/paper/2309.06824/) (2023)
- [Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging](https://mlsanity.com/paper/2208.12737/) (2022)
- [Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration](https://mlsanity.com/paper/1911.07042/) (2019)
- [Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis](https://mlsanity.com/paper/2411.02372/) (2024)
- [Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering](https://mlsanity.com/paper/2312.06358/) (2023)
- [Generalizing Spatial Transformers to Projective Geometry with Applications to 2D/3D Registration](https://mlsanity.com/paper/2003.10987/) (2020)
- [Anatomy-aware and acquisition-agnostic joint registration with SynthMorph](https://mlsanity.com/paper/2301.11329/) (2023)
- [Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation (POINT2)](https://mlsanity.com/paper/1903.03896/) (2019)
- [Equivariant Filters for Efficient Tracking in 3D Imaging](https://mlsanity.com/paper/2103.10255/) (2021)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [Real-time 2D/3D Registration via CNN Regression](https://mlsanity.com/paper/1507.07505/) (2015)
- [Quantifying white matter hyperintensity and brain volumes in heterogeneous clinical and low-field portable MRI](https://mlsanity.com/paper/2312.05119/) (2023)
- [End-to-end Ultrasound Frame to Volume Registration](https://mlsanity.com/paper/2107.06449/) (2021)
- [Global Multi-modal 2D/3D Registration via Local Descriptors Learning](https://mlsanity.com/paper/2205.03439/) (2022)
- [Self-Supervised 2D/3D Registration for X-Ray to CT Image Fusion](https://mlsanity.com/paper/2210.07611/) (2022)
- [DISA: DIfferentiable Similarity Approximation for Universal Multimodal Registration](https://mlsanity.com/paper/2307.09931/) (2023)
- [SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI](https://mlsanity.com/paper/2312.13534/) (2023)
- [Patch-Based Image Similarity for Intraoperative 2D/3D Pelvis Registration During Periacetabular Osteotomy](https://mlsanity.com/paper/1909.10443/) (2019)
- [Synthesizing Individualized Aging Brains in Health and Disease with Generative Models and Parallel Transport](https://mlsanity.com/paper/2502.21049/) (2025)
- [X-Ray to CT Rigid Registration Using Scene Coordinate Regression](https://mlsanity.com/paper/2311.15087/) (2023)
- [Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration](https://mlsanity.com/paper/2011.07294/) (2020)
- [MAD: Modality Agnostic Distance Measure for Image Registration](https://mlsanity.com/paper/2309.02875/) (2023)
- [Deep Iterative 2D/3D Registration](https://mlsanity.com/paper/2107.10004/) (2021)

## Abstract (arXiv)

Intraoperative 2D/3D registration aligns preoperative CT volumes with intraoperative X-ray or fluoroscopic images and is essential for image-guided interventions. Recent learning-based and differentiable registration methods have shown promising accuracy, especially in patient-specific settings where abundant digitally reconstructed radiographs (DRRs) can be synthesized from the target CT. However, training a separate patient-specific model from scratch for every new patient is computationally inefficient and limits practical deployment. In this work, we propose an efficient patient-specific 2D/3D registration framework based on patient-agnostic synthetic pretraining and spherical similarity learning. The model is first pretrained on synthetic DRRs generated from multiple CT volumes to learn transferable pose-sensitive representations, and is then adapted to a new patient using only a limited number of synthetic projections from the target CT. To improve synthetic-to-real robustness without requiring anatomical labels, we introduce a segmentation-free domain randomization strategy that perturbs image intensity, projection physics, field-of-view, occlusion, and fluoroscopic artifacts. The adapted model provides an initial pose estimate, which is further refined using spherical similarity learning and differentiable Levenberg-Marquardt optimization. Experiments on multiple anatomical datasets evaluate whether patient-agnostic synthetic pretraining can improve the efficiency of patient-specific registration, with particular focus on the trade-off between adaptation cost and registration accuracy. The results demonstrate that patient-agnostic synthetic pretraining can significantly reduce patient-specific training requirements while preserving accurate intraoperative 2D/3D registration.