# Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery

arXiv: https://arxiv.org/abs/2308.02917 · HTML: https://mlsanity.com/paper/2308.02917/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 80 references that are in the map.
- First posted on arXiv 5 Aug 2023.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Viewpoints and Keypoints](https://mlsanity.com/paper/1411.6067/) (2014)
- [The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspective](https://mlsanity.com/paper/2108.02238/) (2021)
- [STTAR: Surgical Tool Tracking using off-the-shelf Augmented Reality Head-Mounted Displays](https://mlsanity.com/paper/2208.08880/) (2022)
- [Utility of Optical See-Through Head Mounted Displays in Augmented Reality-Assisted Surgery: A systematic review](https://mlsanity.com/paper/2202.04141/) (2022)
- [Mapping and Localization from Planar Markers](https://mlsanity.com/paper/1606.00151/) (2016)

## Built on by (most cited first, 11 of 11)

- [Creating a Digital Twin of Spinal Surgery: A Proof of Concept](https://mlsanity.com/paper/2403.16736/) (2024)
- [A Review on Organ Deformation Modeling Approaches for Reliable Surgical Navigation using Augmented Reality](https://mlsanity.com/paper/2408.02713/) (2024)
- [Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data](https://mlsanity.com/paper/2401.16027/) (2024)
- [Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting](https://mlsanity.com/paper/2602.18314/) (2026)
- [Towards Markerless Intraoperative Tracking of Deformable Spine Tissue](https://mlsanity.com/paper/2506.23657/) (2025)
- [SurgPointTransformer: Vertebrae Shape Completion with RGB-D Data](https://mlsanity.com/paper/2410.01443/) (2024)
- [Rethinking Rotation-Invariant Recognition of Fine-grained Shapes from the Perspective of Contour Points](https://mlsanity.com/paper/2503.10992/) (2025)
- [NeuralBoneReg: An Instance-Specific Label-Free Point Cloud-Based Method for Multi-Modal Bone Surface Registration](https://mlsanity.com/paper/2511.14286/) (2025)
- [End2Reg: Learning Task-Specific Segmentation for Markerless Registration in Spine Surgery](https://mlsanity.com/paper/2512.13402/) (2025)
- [Unsupervised Deformable Image Registration with Local-Global Attention and Image Decomposition](https://mlsanity.com/paper/2601.14337/) (2026)
- [Multi-Camera AR Guidance System for Surgical Instrument Handling and Assembly: Investigating Workload and Efficiency](https://mlsanity.com/paper/2606.04992/) (2026)

## Abstract (arXiv)

Established surgical navigation systems for pedicle screw placement have been proven to be accurate, but still reveal limitations in registration or surgical guidance. Registration of preoperative data to the intraoperative anatomy remains a time-consuming, error-prone task that includes exposure to harmful radiation. Surgical guidance through conventional displays has well-known drawbacks, as information cannot be presented in-situ and from the surgeon's perspective. Consequently, radiation-free and more automatic registration methods with subsequent surgeon-centric navigation feedback are desirable. In this work, we present an approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models, which then is refined for each vertebra individually and updated in real-time with GPU acceleration while handling surgeon occlusions. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system. The registration method was verified on a public dataset with a mean of 96% successful registrations, a target registration error of 2.73 mm, a screw trajectory error of 1.79deg and a screw entry point error of 2.43 mm. Additionally, the whole pipeline was validated in an ex-vivo surgery, yielding a 100% screw accuracy and a registration accuracy of 1.20 mm. Our results meet clinical demands and emphasize the potential of RGB-D data for fully automatic registration approaches in combination with augmented reality guidance.