# Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy

arXiv: https://arxiv.org/abs/2601.22026 · HTML: https://mlsanity.com/paper/2601.22026/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 67 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 30 of 30)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views](https://mlsanity.com/paper/2505.23716/) (2025)
- [Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians](https://mlsanity.com/paper/2403.17898/) (2024)
- [Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians](https://mlsanity.com/paper/2403.14166/) (2024)
- [Taming 3DGS: High-Quality Radiance Fields with Limited Resources](https://mlsanity.com/paper/2406.15643/) (2024)
- [StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering](https://mlsanity.com/paper/2402.00525/) (2024)
- [MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2405.12218/) (2024)
- [3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods](https://mlsanity.com/paper/2407.09510/) (2024)
- [On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy](https://mlsanity.com/paper/2402.00752/) (2024)
- [Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting](https://mlsanity.com/paper/2503.16979/) (2025)
- [On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images](https://mlsanity.com/paper/2506.05558/) (2025)
- [VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points](https://mlsanity.com/paper/2410.17932/) (2024)
- [MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering](https://mlsanity.com/paper/2407.00435/) (2024)
- [Application of 3D Gaussian Splatting for Cinematic Anatomy on Consumer Class Devices](https://mlsanity.com/paper/2404.11285/) (2024)
- [VRSplat: Fast and Robust Gaussian Splatting for Virtual Reality](https://mlsanity.com/paper/2505.10144/) (2025)
- [EndoSparse: Real-Time Sparse View Synthesis of Endoscopic Scenes using Gaussian Splatting](https://mlsanity.com/paper/2407.01029/) (2024)
- [Semantics-Controlled Gaussian Splatting for Outdoor Scene Reconstruction and Rendering in Virtual Reality](https://mlsanity.com/paper/2409.15959/) (2024)
- [CL-Splats: Continual Learning of Gaussian Splatting with Local Optimization](https://mlsanity.com/paper/2506.21117/) (2025)
- [Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization](https://mlsanity.com/paper/2410.16978/) (2024)
- [Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned and Addressed for XR Research](https://mlsanity.com/paper/2508.04326/) (2025)
- [Real-Time Denoising of Volumetric Path Tracing for Direct Volume Rendering](https://mlsanity.com/paper/2106.08034/) (2021)
- [Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering](https://mlsanity.com/paper/2505.17338/) (2025)
- [A Distributed, Decoupled System for Losslessly Streaming Dynamic Light Probes to Thin Clients](https://mlsanity.com/paper/2103.05875/) (2021)
- [ClipGS: Clippable Gaussian Splatting for Interactive Cinematic Visualization of Volumetric Medical Data](https://mlsanity.com/paper/2507.06647/) (2025)
- [Towards Understanding Depth Perception in Foveated Rendering](https://mlsanity.com/paper/2501.18635/) (2025)
- [From Screen to Space: Evaluating Siemens' Cinematic Reality](https://mlsanity.com/paper/2506.04972/) (2025)

## Built on by (most cited first, 1 of 1)

- [Supercharging Thermal Gaussian Splatting with Depth Estimation](https://mlsanity.com/paper/2605.30328/) (2026)

## Abstract (arXiv)

Volumetric medical imaging offers great potential for understanding complex pathologies. Yet, traditional 2D slices provide little support for interpreting spatial relationships, forcing users to mentally reconstruct anatomy into three dimensions. Direct volumetric path tracing and VR rendering can improve perception but are computationally expensive, while precomputed representations, like Gaussian Splatting, require planning ahead. Both approaches limit interactive use. We propose a hybrid rendering approach for high-quality, interactive, and immersive anatomical visualization. Our method combines streamed foveated path tracing with a lightweight Gaussian Splatting approximation of the periphery. The peripheral model generation is optimized with volume data and continuously refined using foveal renderings, enabling interactive updates. Depth-guided reprojection further improves robustness to latency and allows users to balance fidelity with refresh rate. We compare our method against direct path tracing and Gaussian Splatting. Our results highlight how their combination can preserve strengths in visual quality while re-generating the peripheral model in under a second, eliminating extensive preprocessing and approximations. This opens new options for interactive medical visualization.