# Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization

arXiv: https://arxiv.org/abs/2410.16978 · HTML: https://mlsanity.com/paper/2410.16978/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 70 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024)
- [Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians](https://mlsanity.com/paper/2403.14166/) (2024)
- [HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2403.14530/) (2024)
- [EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS](https://mlsanity.com/paper/2312.04564/) (2023)
- [VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality](https://mlsanity.com/paper/2401.16663/) (2024)
- [Compact 3D Scene Representation via Self-Organizing Gaussian Grids](https://mlsanity.com/paper/2312.13299/) (2023)
- [CompGS: Smaller and Faster Gaussian Splatting with Vector Quantization](https://mlsanity.com/paper/2311.18159/) (2023)
- [FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality](https://mlsanity.com/paper/2103.16365/) (2021)
- [3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods](https://mlsanity.com/paper/2407.09510/) (2024)
- [Compressing Volumetric Radiance Fields to 1 MB](https://mlsanity.com/paper/2211.16386/) (2022)
- [Learning Multi-Object Dynamics with Compositional Neural Radiance Fields](https://mlsanity.com/paper/2202.11855/) (2022)
- [RT-NeRF: Real-Time On-Device Neural Radiance Fields Towards Immersive AR/VR Rendering](https://mlsanity.com/paper/2212.01120/) (2022)
- [CLNeRF: Continual Learning Meets NeRF](https://mlsanity.com/paper/2308.14816/) (2023)
- [EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction](https://mlsanity.com/paper/2401.12561/) (2024)
- [LGS: A Light-weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction](https://mlsanity.com/paper/2406.16073/) (2024)
- [Application of 3D Gaussian Splatting for Cinematic Anatomy on Consumer Class Devices](https://mlsanity.com/paper/2404.11285/) (2024)
- [Textured-GS: Gaussian Splatting with Spatially Defined Color and Opacity](https://mlsanity.com/paper/2407.09733/) (2024)
- [HFGS: 4D Gaussian Splatting with Emphasis on Spatial and Temporal High-Frequency Components for Endoscopic Scene Reconstruction](https://mlsanity.com/paper/2405.17872/) (2024)
- [MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields](https://mlsanity.com/paper/2212.08328/) (2022)
- [SurgicalGaussian: Deformable 3D Gaussians for High-Fidelity Surgical Scene Reconstruction](https://mlsanity.com/paper/2407.05023/) (2024)

## Built on by (most cited first, 7 of 7)

- [InnerGS: Internal Scenes Reconstruction and Segmentation via Factorized 3D Gaussian Splatting](https://mlsanity.com/paper/2508.13287/) (2025)
- [ClipGS: Clippable Gaussian Splatting for Interactive Cinematic Visualization of Volumetric Medical Data](https://mlsanity.com/paper/2507.06647/) (2025)
- [CoRe-GS: Coarse-to-Refined Gaussian Splatting with Semantic Object Focus](https://mlsanity.com/paper/2509.04859/) (2025)
- [Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy](https://mlsanity.com/paper/2601.22026/) (2026)
- [XClipGS: Exact Half-Space Clipping for Medical Volume Gaussian Splatting](https://mlsanity.com/paper/2608.07760/) (2026)
- [MUA: Mobile Ultra-detailed Animatable Avatars](https://mlsanity.com/paper/2604.18583/) (2026)
- [Gaussian Volumetric Representation for Efficient Shear-Warp Visualization](https://mlsanity.com/paper/2607.25377/) (2026)

## Abstract (arXiv)

In medical image visualization, path tracing of volumetric medical data like CT scans produces lifelike three-dimensional visualizations. Immersive VR displays can further enhance the understanding of complex anatomies. Going beyond the diagnostic quality of traditional 2D slices, they enable interactive 3D evaluation of anatomies, supporting medical education and planning. Rendering high-quality visualizations in real-time, however, is computationally intensive and impractical for compute-constrained devices like mobile headsets. We propose a novel approach utilizing GS to create an efficient but static intermediate representation of CT scans. We introduce a layered GS representation, incrementally including different anatomical structures while minimizing overlap and extending the GS training to remove inactive Gaussians. We further compress the created model with clustering across layers. Our approach achieves interactive frame rates while preserving anatomical structures, with quality adjustable to the target hardware. Compared to standard GS, our representation retains some of the explorative qualities initially enabled by immersive path tracing. Selective activation and clipping of layers are possible at rendering time, adding a degree of interactivity to otherwise static GS models. This could enable scenarios where high computational demands would otherwise prohibit using path-traced medical volumes.