# Twin-S: A Digital Twin for Skull-base Surgery

arXiv: https://arxiv.org/abs/2211.11863 · HTML: https://mlsanity.com/paper/2211.11863/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 37 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 5 of 5)

- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [On the Sins of Image Synthesis Loss for Self-supervised Depth Estimation](https://mlsanity.com/paper/2109.06163/) (2021)
- [Virtual Reality for Synergistic Surgical Training and Data Generation](https://mlsanity.com/paper/2111.08097/) (2021)
- [Anatomical Mesh-Based Virtual Fixtures for Surgical Robots](https://mlsanity.com/paper/2006.02415/) (2020)

## Abstract (arXiv)

Purpose: Digital twins are virtual interactive models of the real world, exhibiting identical behavior and properties. In surgical applications, computational analysis from digital twins can be used, for example, to enhance situational awareness. Methods: We present a digital twin framework for skull-base surgeries, named Twin-S, which can be integrated within various image-guided interventions seamlessly. Twin-S combines high-precision optical tracking and real-time simulation. We rely on rigorous calibration routines to ensure that the digital twin representation precisely mimics all real-world processes. Twin-S models and tracks the critical components of skull-base surgery, including the surgical tool, patient anatomy, and surgical camera. Significantly, Twin-S updates and reflects real-world drilling of the anatomical model in frame rate. Results: We extensively evaluate the accuracy of Twin-S, which achieves an average 1.39 mm error during the drilling process. We further illustrate how segmentation masks derived from the continuously updated digital twin can augment the surgical microscope view in a mixed reality setting, where bone requiring ablation is highlighted to provide surgeons additional situational awareness. Conclusion: We present Twin-S, a digital twin environment for skull-base surgery. Twin-S tracks and updates the virtual model in real-time given measurements from modern tracking technologies. Future research on complementing optical tracking with higher-precision vision-based approaches may further increase the accuracy of Twin-S.