# Monocular Microscope to CT Registration using Pose Estimation of the Incus for Augmented Reality Cochlear Implant Surgery

arXiv: https://arxiv.org/abs/2403.07219 · HTML: https://mlsanity.com/paper/2403.07219/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 9 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 2 of 2)

- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Self-supervised Registration and Segmentation of the Ossicles with A Single Ground Truth Label](https://mlsanity.com/paper/2302.07967/) (2023)

## Built on by (most cited first, 7 of 7)

- [Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment Anything Model 2](https://mlsanity.com/paper/2408.01648/) (2024)
- [Self-supervised Mamba-based Mastoidectomy Shape Prediction for Cochlear Implant Surgery](https://mlsanity.com/paper/2407.15787/) (2024)
- [Post-mastoidectomy Surface Multi-View Synthesis from a Single Microscopy Image](https://mlsanity.com/paper/2409.03190/) (2024)
- [SSDD-GAN: Single-Step Denoising Diffusion GAN for Cochlear Implant Surgical Scene Completion](https://mlsanity.com/paper/2502.05710/) (2025)
- [Vision6D: 3D-to-2D Interactive Visualization and Annotation Tool for 6D Pose Estimation](https://mlsanity.com/paper/2504.15329/) (2025)
- [Weakly-supervised Mamba-Based Mastoidectomy Shape Prediction for Cochlear Implant Surgery Using 3D T-Distribution Loss](https://mlsanity.com/paper/2505.18368/) (2025)
- [Monocular Marker-free Patient-to-Image Intraoperative Registration for Cochlear Implant Surgery](https://mlsanity.com/paper/2505.18381/) (2025)

## Abstract (arXiv)

For those experiencing severe-to-profound sensorineural hearing loss, the cochlear implant (CI) is the preferred treatment. Augmented reality (AR) aided surgery can potentially improve CI procedures and hearing outcomes. Typically, AR solutions for image-guided surgery rely on optical tracking systems to register pre-operative planning information to the display so that hidden anatomy or other important information can be overlayed and co-registered with the view of the surgical scene. In this paper, our goal is to develop a method that permits direct 2D-to-3D registration of the microscope video to the pre-operative Computed Tomography (CT) scan without the need for external tracking equipment. Our proposed solution involves using surface mapping of a portion of the incus in surgical recordings and determining the pose of this structure relative to the surgical microscope by performing pose estimation via the perspective-n-point (PnP) algorithm. This registration can then be applied to pre-operative segmentations of other anatomy-of-interest, as well as the planned electrode insertion trajectory to co-register this information for the AR display. Our results demonstrate the accuracy with an average rotation error of less than 25 degrees and a translation error of less than 2 mm, 3 mm, and 0.55% for the x, y, and z axes, respectively. Our proposed method has the potential to be applicable and generalized to other surgical procedures while only needing a monocular microscope during intra-operation.