# Post-mastoidectomy Surface Multi-View Synthesis from a Single Microscopy Image

arXiv: https://arxiv.org/abs/2409.03190 · HTML: https://mlsanity.com/paper/2409.03190/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 10 references that are in the map.
- First posted on arXiv 31 Aug 2024.

## Builds on (most cited first, 4 of 4)

- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Monocular Microscope to CT Registration using Pose Estimation of the Incus for Augmented Reality Cochlear Implant Surgery](https://mlsanity.com/paper/2403.07219/) (2024)
- [Self-supervised Registration and Segmentation of the Ossicles with A Single Ground Truth Label](https://mlsanity.com/paper/2302.07967/) (2023)

## Built on by (most cited first, 4 of 4)

- [SSDD-GAN: Single-Step Denoising Diffusion GAN for Cochlear Implant Surgical Scene Completion](https://mlsanity.com/paper/2502.05710/) (2025)
- [Vision6D: 3D-to-2D Interactive Visualization and Annotation Tool for 6D Pose Estimation](https://mlsanity.com/paper/2504.15329/) (2025)
- [Weakly-supervised Mamba-Based Mastoidectomy Shape Prediction for Cochlear Implant Surgery Using 3D T-Distribution Loss](https://mlsanity.com/paper/2505.18368/) (2025)
- [Monocular Marker-free Patient-to-Image Intraoperative Registration for Cochlear Implant Surgery](https://mlsanity.com/paper/2505.18381/) (2025)

## Abstract (arXiv)

Cochlear Implant (CI) procedures involve performing an invasive mastoidectomy to insert an electrode array into the cochlea. In this paper, we introduce a novel pipeline that is capable of generating synthetic multi-view videos from a single CI microscope image. In our approach, we use a patient's pre-operative CT scan to predict the post-mastoidectomy surface using a method designed for this purpose. We manually align the surface with a selected microscope frame to obtain an accurate initial pose of the reconstructed CT mesh relative to the microscope. We then perform UV projection to transfer the colors from the frame to surface textures. Novel views of the textured surface can be used to generate a large dataset of synthetic frames with ground truth poses. We evaluated the quality of synthetic views rendered using Pytorch3D and PyVista. We found both rendering engines lead to similarly high-quality synthetic novel-view frames compared to ground truth with a structural similarity index for both methods averaging about 0.86. A large dataset of novel views with known poses is critical for ongoing training of a method to automatically estimate microscope pose for 2D to 3D registration with the pre-operative CT to facilitate augmented reality surgery. This dataset will empower various downstream tasks, such as integrating Augmented Reality (AR) in the OR, tracking surgical tools, and supporting other video analysis studies.