# Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study

arXiv: https://arxiv.org/abs/2511.02086 · HTML: https://mlsanity.com/paper/2511.02086/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 3 Nov 2025.

## Builds on (most cited first, 6 of 6)

- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [STTAR: Surgical Tool Tracking using off-the-shelf Augmented Reality Head-Mounted Displays](https://mlsanity.com/paper/2208.08880/) (2022)
- [Utility of Optical See-Through Head Mounted Displays in Augmented Reality-Assisted Surgery: A systematic review](https://mlsanity.com/paper/2202.04141/) (2022)
- [EVD Surgical Guidance with Retro-Reflective Tool Tracking and Spatial Reconstruction using Head-Mounted Augmented Reality Device](https://mlsanity.com/paper/2306.15490/) (2023)
- [EasyREG: Easy Depth-Based Markerless Registration and Tracking using Augmented Reality Device for Surgical Guidance](https://mlsanity.com/paper/2504.09498/) (2025)

## Built on by (most cited first, 2 of 2)

- [All-in-One Augmented Reality Guided Head and Neck Tumor Resection](https://mlsanity.com/paper/2603.29495/) (2026)
- [Toward surface-based registration of a virtual preoperative cutting guide onto the mandible for reconstruction surgery](https://mlsanity.com/paper/2608.06599/) (2026)

## Abstract (arXiv)

Purpose: In this paper, we develop and clinically evaluate a depth-only, markerless augmented reality (AR) registration pipeline on a head-mounted display, and assess accuracy across small or low-curvature anatomies in real-life operative settings. Methods: On HoloLens 2, we align Articulated HAnd Tracking (AHAT) depth to Computed Tomography (CT)-derived skin meshes via (i) depth-bias correction, (ii) brief human-in-the-loop initialization, (iii) global and local registration. We validated the surface-tracing error metric by comparing "skin-to-bone" relative distances to CT ground truth on leg and foot models, using an AR-tracked tool. We then performed seven intraoperative target trials (feet x2, ear x3, leg x2) during the initial stage of fibula free-flap harvest and mandibular reconstruction surgery, and collected 500+ data per trial. Results: Preclinical validation showed tight agreement between AR-traced and CT distances (leg: median |Delta d| 0.78 mm, RMSE 0.97 mm; feet: 0.80 mm, 1.20 mm). Clinically, per-point error had a median of 3.9 mm. Median errors by anatomy were 3.2 mm (feet), 4.3 mm (ear), and 5.3 mm (lower leg), with 5 mm coverage 92-95%, 84-90%, and 72-86%, respectively. Feet vs. lower leg differed significantly (Delta median ~1.1 mm; p < 0.001). Conclusion: A depth-only, markerless AR pipeline on HMDs achieved ~3-4 mm median error across feet, ear, and lower leg in live surgical settings without fiducials, approaching typical clinical error thresholds for moderate-risk tasks. Human-guided initialization plus global-to-local registration enabled accurate alignment on small or low-curvature targets, improving the clinical readiness of markerless AR guidance.