# Deep Learning-based Point Cloud Registration for Augmented Reality-guided Surgery

arXiv: https://arxiv.org/abs/2405.03314 · HTML: https://mlsanity.com/paper/2405.03314/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 9 of 9)

- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences](https://mlsanity.com/paper/2005.01014/) (2020)
- [Robust Point Cloud Registration Framework Based on Deep Graph Matching](https://mlsanity.com/paper/2103.04256/) (2021)
- [A comprehensive survey on point cloud registration](https://mlsanity.com/paper/2103.02690/) (2021)
- [DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration](https://mlsanity.com/paper/1905.04153/) (2019)
- [High-dimensional Convolutional Networks for Geometric Pattern Recognition](https://mlsanity.com/paper/2005.08144/) (2020)
- [The HoloLens in Medicine: A systematic Review and Taxonomy](https://mlsanity.com/paper/2209.03245/) (2022)
- [On-the-fly Augmented Reality for Orthopaedic Surgery Using a Multi-Modal Fiducial](https://mlsanity.com/paper/1801.01560/) (2018)

## Built on by (most cited first, 6 of 6)

- [SemICP: Semantic Non-Rigid Point Cloud Registration with Elastic Energy Regularization](https://mlsanity.com/paper/2503.00972/) (2025)
- [Overview and Comparison of AVS Point Cloud Compression Standard](https://mlsanity.com/paper/2602.08613/) (2026)
- [DINE: Distance Is Not Enough -- Learning Global Deformation Priors for Robust Soft-Tissue Point Cloud Registration](https://mlsanity.com/paper/2607.14946/) (2026)
- [SM3D: Mitigating Spectral Bias and Semantic Dilution in Point Cloud State Space Models](https://mlsanity.com/paper/2505.11099/) (2025)
- [NeuralBoneReg: An Instance-Specific Label-Free Point Cloud-Based Method for Multi-Modal Bone Surface Registration](https://mlsanity.com/paper/2511.14286/) (2025)
- [End2Reg: Learning Task-Specific Segmentation for Markerless Registration in Spine Surgery](https://mlsanity.com/paper/2512.13402/) (2025)

## Abstract (arXiv)

Point cloud registration aligns 3D point clouds using spatial transformations. It is an important task in computer vision, with applications in areas such as augmented reality (AR) and medical imaging. This work explores the intersection of two research trends: the integration of AR into image-guided surgery and the use of deep learning for point cloud registration. The main objective is to evaluate the feasibility of applying deep learning-based point cloud registration methods for image-to-patient registration in augmented reality-guided surgery. We created a dataset of point clouds from medical imaging and corresponding point clouds captured with a popular AR device, the HoloLens 2. We evaluate three well-established deep learning models in registering these data pairs. While we find that some deep learning methods show promise, we show that a conventional registration pipeline still outperforms them on our challenging dataset.