# AI-Enhanced Virtual Reality in Medicine: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2402.03093 · HTML: https://mlsanity.com/paper/2402.03093/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 65 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos](https://mlsanity.com/paper/2109.03223/) (2021)
- [Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery](https://mlsanity.com/paper/2206.15255/) (2022)
- [EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos](https://mlsanity.com/paper/2307.11307/) (2023)
- [Diffusion Deformable Model for 4D Temporal Medical Image Generation](https://mlsanity.com/paper/2206.13295/) (2022)
- [MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimation](https://mlsanity.com/paper/1808.08180/) (2018)
- [4D-OR: Semantic Scene Graphs for OR Domain Modeling](https://mlsanity.com/paper/2203.11937/) (2022)
- [SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgery](https://mlsanity.com/paper/2304.09974/) (2023)
- [Surgical-VQLA: Transformer with Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery](https://mlsanity.com/paper/2305.11692/) (2023)
- [IntrA: 3D Intracranial Aneurysm Dataset for Deep Learning](https://mlsanity.com/paper/2003.02920/) (2020)
- [D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2201.00462/) (2022)
- [Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery](https://mlsanity.com/paper/2006.16166/) (2020)
- [CuNeRF: Cube-Based Neural Radiance Field for Zero-Shot Medical Image Arbitrary-Scale Super Resolution](https://mlsanity.com/paper/2303.16242/) (2023)
- [Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies](https://mlsanity.com/paper/2012.04872/) (2020)
- [Two-Stage Mesh Deep Learning for Automated Tooth Segmentation and Landmark Localization on 3D Intraoral Scans](https://mlsanity.com/paper/2109.11941/) (2021)
- [Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videos](https://mlsanity.com/paper/2111.11044/) (2021)
- [Dynamic Scene Graph Representation for Surgical Video](https://mlsanity.com/paper/2309.14538/) (2023)
- [Point Cloud Diffusion Models for Automatic Implant Generation](https://mlsanity.com/paper/2303.08061/) (2023)
- [CAT-ViL: Co-Attention Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery](https://mlsanity.com/paper/2307.05182/) (2023)
- [TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation](https://mlsanity.com/paper/2012.13697/) (2020)
- [Electrocardio Panorama: Synthesizing New ECG Views with Self-supervision](https://mlsanity.com/paper/2105.06293/) (2021)
- [3DTeethSeg'22: 3D Teeth Scan Segmentation and Labeling Challenge](https://mlsanity.com/paper/2305.18277/) (2023)
- [POV-Surgery: A Dataset for Egocentric Hand and Tool Pose Estimation During Surgical Activities](https://mlsanity.com/paper/2307.10387/) (2023)
- [MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation](https://mlsanity.com/paper/2205.04846/) (2022)
- [DMCVR: Morphology-Guided Diffusion Model for 3D Cardiac Volume Reconstruction](https://mlsanity.com/paper/2308.09223/) (2023)
- [Patient-specific virtual spine straightening and vertebra inpainting: An automatic framework for osteoplasty planning](https://mlsanity.com/paper/2103.07279/) (2021)

## Built on by (most cited first, 2 of 2)

- [TeleOR: Real-time Telemedicine System for Full-Scene Operating Room](https://mlsanity.com/paper/2407.19763/) (2024)
- [CPSL: Representing Volumetric Video via Content-Promoted Scene Layers](https://mlsanity.com/paper/2511.14927/) (2025)

## Abstract (arXiv)

With the rapid advance of computer graphics and artificial intelligence technologies, the ways we interact with the world have undergone a transformative shift. Virtual Reality (VR) technology, aided by artificial intelligence (AI), has emerged as a dominant interaction media in multiple application areas, thanks to its advantage of providing users with immersive experiences. Among those applications, medicine is considered one of the most promising areas. In this paper, we present a comprehensive examination of the burgeoning field of AI-enhanced VR applications in medical care and services. By introducing a systematic taxonomy, we meticulously classify the pertinent techniques and applications into three well-defined categories based on different phases of medical diagnosis and treatment: Visualization Enhancement, VR-related Medical Data Processing, and VR-assisted Intervention. This categorization enables a structured exploration of the diverse roles that AI-powered VR plays in the medical domain, providing a framework for a more comprehensive understanding and evaluation of these technologies. To our best knowledge, this is the first systematic survey of AI-powered VR systems in medical settings, laying a foundation for future research in this interdisciplinary domain.