# VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points

arXiv: https://arxiv.org/abs/2410.17932 · HTML: https://mlsanity.com/paper/2410.17932/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 136 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 30 of 69)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs](https://mlsanity.com/paper/2112.10703/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](https://mlsanity.com/paper/2103.03231/) (2021)

## Built on by (most cited first, 29 of 29)

- [Deformable Beta Splatting](https://mlsanity.com/paper/2501.18630/) (2025)
- [Faster-GS: Analyzing and Improving Gaussian Splatting Optimization](https://mlsanity.com/paper/2602.09999/) (2026)
- [Perceptual-GS: Scene-adaptive Perceptual Densification for Gaussian Splatting](https://mlsanity.com/paper/2506.12400/) (2025)
- [From Coarse to Fine: Learnable Discrete Wavelet Transforms for Efficient 3D Gaussian Splatting](https://mlsanity.com/paper/2506.23042/) (2025)
- [Multi-Spectral Gaussian Splatting with Neural Color Representation](https://mlsanity.com/paper/2506.03407/) (2025)
- [GS-Scale: Unlocking Large-Scale 3D Gaussian Splatting Training via Host Offloading](https://mlsanity.com/paper/2509.15645/) (2025)
- [OracleGS: Grounding Generative Priors for Sparse-View Gaussian Splatting](https://mlsanity.com/paper/2509.23258/) (2025)
- [Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generation](https://mlsanity.com/paper/2603.23491/) (2026)
- [Towards Integrating Multi-Spectral Imaging with Gaussian Splatting](https://mlsanity.com/paper/2509.00989/) (2025)
- [ControlGS: Consistent Structural Compression Control for Deployment-Aware Gaussian Splatting](https://mlsanity.com/paper/2505.10473/) (2025)
- [GS-Verse: Mesh-based Gaussian Splatting for Physics-aware Interaction in Virtual Reality](https://mlsanity.com/paper/2510.11878/) (2025)
- [DAV-GSWT: Diffusion-Active-View Sampling for Data-Efficient Gaussian Splatting Wang Tiles](https://mlsanity.com/paper/2602.15355/) (2026)
- [NanoGS: Training-Free Gaussian Splat Simplification](https://mlsanity.com/paper/2603.16103/) (2026)
- [A Bag of Tricks for Efficient Implicit Neural Point Clouds](https://mlsanity.com/paper/2508.19140/) (2025)
- [From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representations](https://mlsanity.com/paper/2509.23555/) (2025)
- [DynamicTree: Interactive Real Tree Animation via Sparse Voxel Spectrum](https://mlsanity.com/paper/2510.22213/) (2025)
- [The Impact and Outlook of 3D Gaussian Splatting](https://mlsanity.com/paper/2510.26694/) (2025)
- [SurfFill: Completion of LiDAR Point Clouds via Gaussian Surfel Splatting](https://mlsanity.com/paper/2512.03010/) (2025)
- [Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy](https://mlsanity.com/paper/2601.22026/) (2026)
- [Splaxel: Efficient Distributed Training of 3D Gaussian Splatting for Large-scale Scene Reconstruction via Pixel-level Communication](https://mlsanity.com/paper/2606.18588/) (2026)
- [GaussLite: Online Task-Conditioned 3D Gaussian Splatting for Real-Time Robotic Mapping](https://mlsanity.com/paper/2606.30809/) (2026)
- [DeGS: A Scalable 3DGS Architecture via Decoupled Workload Parsing and Reorganization](https://mlsanity.com/paper/2608.02099/) (2026)
- [EyeNavGS: A 6-DoF Navigation Dataset and Record-n-Replay Software for Real-World 3DGS Scenes in VR](https://mlsanity.com/paper/2506.02380/) (2025)
- [Enhancing Novel View Synthesis from extremely sparse views with SfM-free 3D Gaussian Splatting Framework](https://mlsanity.com/paper/2508.15457/) (2025)
- [SwiftGS: Episodic Priors for Immediate Satellite Surface Recovery](https://mlsanity.com/paper/2603.18634/) (2026)
- [GA-GS: Generation-Assisted Gaussian Splatting for Static Scene Reconstruction](https://mlsanity.com/paper/2604.04331/) (2026)
- [3DGS3: Joint Super Sampling and Frame Interpolation for Real-Time Large-Scale 3DGS Rendering](https://mlsanity.com/paper/2605.11489/) (2026)
- [Seeing What Matters: Perceptual Wrapper with Common Randomness for 3D Gaussian Splatting](https://mlsanity.com/paper/2606.11782/) (2026)
- [ControlGS: Conditioning Neural Gaussians for Downstream-Processing-Aware XR Rendering](https://mlsanity.com/paper/2609.32038/) (2026)

## Abstract (arXiv)

Recent advances in novel view synthesis have demonstrated impressive results in fast photorealistic scene rendering through differentiable point rendering, either via Gaussian Splatting (3DGS) [Kerbl and Kopanas et al. 2023] or neural point rendering [Aliev et al. 2020]. Unfortunately, these directions require either a large number of small Gaussians or expensive per-pixel post-processing for reconstructing fine details, which negatively impacts rendering performance. To meet the high performance demands of virtual reality (VR) systems, primitive or pixel counts therefore must be kept low, affecting visual quality. In this paper, we propose a novel hybrid approach based on foveated rendering as a promising solution that combines the strengths of both point rendering directions regarding performance sweet spots. Analyzing the compatibility with the human visual system, we find that using a low-detailed, few primitive smooth Gaussian representation for the periphery is cheap to compute and meets the perceptual demands of peripheral vision. For the fovea only, we use neural points with a convolutional neural network for the small pixel footprint, which provides sharp, detailed output within the rendering budget. This combination also allows for synergistic method accelerations with point occlusion culling and reducing the demands on the neural network. Our evaluation confirms that our approach increases sharpness and details compared to a standard VR-ready 3DGS configuration, and participants of a user study overwhelmingly preferred our method. Our system meets the necessary performance requirements for real-time VR interactions, ultimately enhancing the user's immersive experience. The project page can be found at: https://lfranke.github.io/vr_splatting