# FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks

arXiv: https://arxiv.org/abs/2209.09965 · HTML: https://mlsanity.com/paper/2209.09965/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 68 references that are in the map.
- First posted on arXiv 20 Sep 2022.

## Builds on (most cited first, 7 of 7)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [Learning Adaptive Sampling and Reconstruction for Volume Visualization](https://mlsanity.com/paper/2007.10093/) (2020)
- [Deep Volumetric Ambient Occlusion](https://mlsanity.com/paper/2008.08345/) (2020)

## Built on by (most cited first, 7 of 7)

- [HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation](https://mlsanity.com/paper/2304.04188/) (2023)
- [Regularized Multi-Decoder Ensemble for an Error-Aware Scene Representation Network](https://mlsanity.com/paper/2407.19082/) (2024)
- [GSCache: Real-Time Radiance Caching for Volume Path Tracing using 3D Gaussian Splatting](https://mlsanity.com/paper/2507.19718/) (2025)
- [F-Hash: Feature-Based Hash Design for Time-Varying Volume Visualization via Multi-Resolution Tesseract Encoding](https://mlsanity.com/paper/2507.03836/) (2025)
- [Photon Field Networks for Dynamic Real-Time Volumetric Global Illumination](https://mlsanity.com/paper/2304.07338/) (2023)
- [FovealNet: Advancing AI-Driven Gaze Tracking Solutions for Optimized Foveated Rendering System Performance in Virtual Reality](https://mlsanity.com/paper/2412.10456/) (2024)
- [Fixed-Budget Gaussian Volume Encoding with Structure-Aware Allocation](https://mlsanity.com/paper/2608.14112/) (2026)

## Abstract (arXiv)

Volume data is found in many important scientific and engineering applications. Rendering this data for visualization at high quality and interactive rates for demanding applications such as virtual reality is still not easily achievable even using professional-grade hardware. We introduce FoVolNet -- a method to significantly increase the performance of volume data visualization. We develop a cost-effective foveated rendering pipeline that sparsely samples a volume around a focal point and reconstructs the full-frame using a deep neural network. Foveated rendering is a technique that prioritizes rendering computations around the user's focal point. This approach leverages properties of the human visual system, thereby saving computational resources when rendering data in the periphery of the user's field of vision. Our reconstruction network combines direct and kernel prediction methods to produce fast, stable, and perceptually convincing output. With a slim design and the use of quantization, our method outperforms state-of-the-art neural reconstruction techniques in both end-to-end frame times and visual quality. We conduct extensive evaluations of the system's rendering performance, inference speed, and perceptual properties, and we provide comparisons to competing neural image reconstruction techniques. Our test results show that FoVolNet consistently achieves significant time saving over conventional rendering while preserving perceptual quality.