# Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration

arXiv: https://arxiv.org/abs/1911.07042 · HTML: https://mlsanity.com/paper/1911.07042/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2019.

## Built on by (most cited first, 29 of 29)

- [Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration](https://mlsanity.com/paper/2206.08903/) (2022)
- [Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering](https://mlsanity.com/paper/2312.06358/) (2023)
- [DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering](https://mlsanity.com/paper/2406.02518/) (2024)
- [The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspective](https://mlsanity.com/paper/2108.02238/) (2021)
- [Self-Supervised 2D/3D Registration for X-Ray to CT Image Fusion](https://mlsanity.com/paper/2210.07611/) (2022)
- [X-Ray to CT Rigid Registration Using Scene Coordinate Regression](https://mlsanity.com/paper/2311.15087/) (2023)
- [RayEmb: Arbitrary Landmark Detection in X-Ray Images Using Ray Embedding Subspace](https://mlsanity.com/paper/2410.08152/) (2024)
- [SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments](https://mlsanity.com/paper/2206.06127/) (2022)
- [Transfer Learning from an Artificial Radiograph-landmark Dataset for Registration of the Anatomic Skull Model to Dual Fluoroscopic X-ray Images](https://mlsanity.com/paper/2108.06466/) (2021)
- [PolyPose: Deformable 2D/3D Registration via Polyrigid Transformations](https://mlsanity.com/paper/2505.19256/) (2025)
- [Deep Learning compatible Differentiable X-ray Projections for Inverse Rendering](https://mlsanity.com/paper/2102.02912/) (2021)
- [Multistep Automated Data Labelling Procedure (MADLaP) for Thyroid Nodules on Ultrasound: An Artificial Intelligence Approach for Automating Image Annotation](https://mlsanity.com/paper/2206.14305/) (2022)
- [Segment Any Medical Model Extended](https://mlsanity.com/paper/2403.18114/) (2024)
- [2D/3D Registration of Acetabular Hip Implants Under Perspective Projection and Fully Differentiable Ellipse Fitting](https://mlsanity.com/paper/2503.07763/) (2025)
- [Automatic Multi-View X-Ray/CT Registration Using Bone Substructure Contours](https://mlsanity.com/paper/2506.13292/) (2025)
- [Human-AI Collaboration and Explainability for 2D/3D Registration Quality Assurance](https://mlsanity.com/paper/2507.17597/) (2025)
- [Intraoperative 2D/3D Registration via Spherical Similarity Learning and Differentiable Levenberg-Marquardt Optimization](https://mlsanity.com/paper/2509.06890/) (2025)
- [LayersReg: A Layer-by-Layer Progressive Regressor for Reliable Intraoperative 3D/2D Registration](https://mlsanity.com/paper/2606.26647/) (2026)
- [The Impact of Loss Functions and Scene Representations for 3D/2D Registration on Single-view Fluoroscopic X-ray Pose Estimation](https://mlsanity.com/paper/2308.00214/) (2023)
- [Rapid patient-specific neural networks for intraoperative X-ray to volume registration](https://mlsanity.com/paper/2503.16309/) (2025)
- [Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints](https://mlsanity.com/paper/2506.22191/) (2025)
- [RadGS-Reg: Registering Spine CT with Biplanar X-rays via Joint 3D Radiative Gaussians Reconstruction and 3D/3D Registration](https://mlsanity.com/paper/2508.21154/) (2025)
- [Veriserum: A dual-plane fluoroscopic dataset with knee implant phantoms for deep learning in medical imaging](https://mlsanity.com/paper/2509.05483/) (2025)
- [NeuralBoneReg: An Instance-Specific Label-Free Point Cloud-Based Method for Multi-Modal Bone Surface Registration](https://mlsanity.com/paper/2511.14286/) (2025)
- [Enhanced Landmark Detection Model in Pelvic Fluoroscopy using 2D/3D Registration Loss](https://mlsanity.com/paper/2511.21575/) (2025)
- [2D Versus 3D Diffusion for In Silico Training of Interventional X-ray AI Models](https://mlsanity.com/paper/2606.21414/) (2026)
- [Learning Dense 2D-3D Correspondence for X-ray-to-CT Registration of Knee Bones](https://mlsanity.com/paper/2607.22803/) (2026)
- [Patient-Agnostic Synthetic Pretraining for Efficient Patient-Specific Intraoperative 2D/3D Registration](https://mlsanity.com/paper/2607.23343/) (2026)
- [FleXray: Universal Clinical X-ray Segmentation](https://mlsanity.com/paper/2609.26756/) (2026)

## Abstract (arXiv)

Fluoroscopy is the standard imaging modality used to guide hip surgery and is therefore a natural sensor for computer-assisted navigation. In order to efficiently solve the complex registration problems presented during navigation, human-assisted annotations of the intraoperative image are typically required. This manual initialization interferes with the surgical workflow and diminishes any advantages gained from navigation. We propose a method for fully automatic registration using annotations produced by a neural network. Neural networks are trained to simultaneously segment anatomy and identify landmarks in fluoroscopy. Training data is obtained using an intraoperatively incompatible 2D/3D registration of hip anatomy. Ground truth 2D labels are established using projected 3D annotations. Intraoperative registration couples an intensity-based strategy with annotations inferred by the network and requires no human assistance. Ground truth labels were obtained in 366 fluoroscopic images across 6 cadaveric specimens. In a leave-one-subject-out experiment, networks obtained mean dice coefficients for left and right hemipelves, left and right femurs of 0.86, 0.87, 0.90, and 0.84. The mean 2D landmark error was 5.0 mm. The pelvis was registered within 1 degree for 86% of the images when using the proposed intraoperative approach with an average runtime of 7 seconds. In comparison, an intensity-only approach without manual initialization, registered the pelvis to 1 degree in 18% of images. We have created the first accurately annotated, non-synthetic, dataset of hip fluoroscopy. By using these annotations as training data for neural networks, state of the art performance in fluoroscopic segmentation and landmark localization was achieved. Integrating these annotations allows for a robust, fully automatic, and efficient intraoperative registration during fluoroscopic navigation of the hip.