# EasyREG: Easy Depth-Based Markerless Registration and Tracking using Augmented Reality Device for Surgical Guidance

arXiv: https://arxiv.org/abs/2504.09498 · HTML: https://mlsanity.com/paper/2504.09498/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 50 references that are in the map.
- First posted on arXiv 13 Apr 2025.

## Builds on (most cited first, 15 of 15)

- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration](https://mlsanity.com/paper/1605.03344/) (2016)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [Fast and Robust Iterative Closest Point](https://mlsanity.com/paper/2007.07627/) (2020)
- [PCRNet: Point Cloud Registration Network using PointNet Encoding](https://mlsanity.com/paper/1908.07906/) (2019)
- [FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization](https://mlsanity.com/paper/1811.10136/) (2018)
- [STTAR: Surgical Tool Tracking using off-the-shelf Augmented Reality Head-Mounted Displays](https://mlsanity.com/paper/2208.08880/) (2022)
- [KSS-ICP: Point Cloud Registration based on Kendall Shape Space](https://mlsanity.com/paper/2211.02807/) (2022)
- [Utility of Optical See-Through Head Mounted Displays in Augmented Reality-Assisted Surgery: A systematic review](https://mlsanity.com/paper/2202.04141/) (2022)
- [Unsupervised Learning of 3D Point Set Registration](https://mlsanity.com/paper/2006.06200/) (2020)
- [EVD Surgical Guidance with Retro-Reflective Tool Tracking and Spatial Reconstruction using Head-Mounted Augmented Reality Device](https://mlsanity.com/paper/2306.15490/) (2023)
- [Navigate Biopsy with Ultrasound under Augmented Reality Device: Towards Higher System Performance](https://mlsanity.com/paper/2402.02414/) (2024)

## Built on by (most cited first, 6 of 6)

- [Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study](https://mlsanity.com/paper/2511.02086/) (2025)
- [Multimodal Feedback for Handheld Tool Guidance: Combining Wrist-Based Haptics with Augmented Reality](https://mlsanity.com/paper/2601.12037/) (2026)
- [ARport: An Augmented Reality System for Markerless Image-Guided Port Placement in Robotic Surgery](https://mlsanity.com/paper/2602.14153/) (2026)
- [Patient4D: Temporally Consistent Patient Body Mesh Recovery from Monocular Operating Room Video](https://mlsanity.com/paper/2603.17178/) (2026)
- [VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection](https://mlsanity.com/paper/2606.24498/) (2026)
- [Toward surface-based registration of a virtual preoperative cutting guide onto the mandible for reconstruction surgery](https://mlsanity.com/paper/2608.06599/) (2026)

## Abstract (arXiv)

The use of Augmented Reality (AR) devices for surgical guidance has gained increasing traction in the medical field. Traditional registration methods often rely on external fiducial markers to achieve high accuracy and real-time performance. However, these markers introduce cumbersome calibration procedures and can be challenging to deploy in clinical settings. While commercial solutions have attempted real-time markerless tracking using the native RGB cameras of AR devices, their accuracy remains questionable for medical guidance, primarily due to occlusions and significant outliers between the live sensor data and the preoperative target anatomy point cloud derived from MRI or CT scans. In this work, we present a markerless framework that relies only on the depth sensor of AR devices and consists of two modules: a registration module for high-precision, outlier-robust target anatomy localization, and a tracking module for real-time pose estimation. The registration module integrates depth sensor error correction, a human-in-the-loop region filtering technique, and a robust global alignment with curvature-aware feature sampling, followed by local ICP refinement, for markerless alignment of preoperative models with patient anatomy. The tracking module employs a fast and robust registration algorithm that uses the initial pose from the registration module to estimate the target pose in real-time. We comprehensively evaluated the performance of both modules through simulation and real-world measurements. The results indicate that our markerless system achieves superior performance for registration and comparable performance for tracking to industrial solutions. The two-module design makes our system a one-stop solution for surgical procedures where the target anatomy moves or stays static during surgery.