# Fully Differentiable Correlation-driven 2D/3D Registration for X-ray to CT Image Fusion

arXiv: https://arxiv.org/abs/2402.02498 · HTML: https://mlsanity.com/paper/2402.02498/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [NestedFormer: Nested Modality-Aware Transformer for Brain Tumor Segmentation](https://mlsanity.com/paper/2208.14876/) (2022)
- [Generalizing Spatial Transformers to Projective Geometry with Applications to 2D/3D Registration](https://mlsanity.com/paper/2003.10987/) (2020)
- [Patch-Based Image Similarity for Intraoperative 2D/3D Pelvis Registration During Periacetabular Osteotomy](https://mlsanity.com/paper/1909.10443/) (2019)
- [Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration](https://mlsanity.com/paper/2011.07294/) (2020)
- [SpineCLUE: Automatic Vertebrae Identification Using Contrastive Learning and Uncertainty Estimation](https://mlsanity.com/paper/2401.07271/) (2024)

## Built on by (most cited first, 3 of 3)

- [An Optimization-based Baseline for Rigid 2D/3D Registration Applied to Spine Surgical Navigation Using CMA-ES](https://mlsanity.com/paper/2402.05642/) (2024)
- [Intraoperative 2D/3D Registration via Spherical Similarity Learning and Differentiable Levenberg-Marquardt Optimization](https://mlsanity.com/paper/2509.06890/) (2025)
- [Patient-Agnostic Synthetic Pretraining for Efficient Patient-Specific Intraoperative 2D/3D Registration](https://mlsanity.com/paper/2607.23343/) (2026)

## Abstract (arXiv)

Image-based rigid 2D/3D registration is a critical technique for fluoroscopic guided surgical interventions. In recent years, some learning-based fully differentiable methods have produced beneficial outcomes while the process of feature extraction and gradient flow transmission still lack controllability and interpretability. To alleviate these problems, in this work, we propose a novel fully differentiable correlation-driven network using a dual-branch CNN-transformer encoder which enables the network to extract and separate low-frequency global features from high-frequency local features. A correlation-driven loss is further proposed for low-frequency feature and high-frequency feature decomposition based on embedded information. Besides, a training strategy that learns to approximate a convex-shape similarity function is applied in our work. We test our approach on a in-house datasetand show that it outperforms both existing fully differentiable learning-based registration approaches and the conventional optimization-based baseline.