# Fast Neural Representations for Direct Volume Rendering

arXiv: https://arxiv.org/abs/2112.01579 · HTML: https://mlsanity.com/paper/2112.01579/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 74 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 30 of 31)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](https://mlsanity.com/paper/2103.03231/) (2021)
- [AutoInt: Automatic Integration for Fast Neural Volume Rendering](https://mlsanity.com/paper/2012.01714/) (2020)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [Real-time Neural Radiance Caching for Path Tracing](https://mlsanity.com/paper/2106.12372/) (2021)
- [NVIDIA Tensor Core Programmability, Performance & Precision](https://mlsanity.com/paper/1803.04014/) (2018)
- [ACORN: Adaptive Coordinate Networks for Neural Scene Representation](https://mlsanity.com/paper/2105.02788/) (2021)
- [Compressive Neural Representations of Volumetric Scalar Fields](https://mlsanity.com/paper/2104.04523/) (2021)
- [TTHRESH: Tensor Compression for Multidimensional Visual Data](https://mlsanity.com/paper/1806.05952/) (2018)
- [A Generative Model for Volume Rendering](https://mlsanity.com/paper/1710.09545/) (2017)
- [Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution](https://mlsanity.com/paper/1906.06520/) (2019)
- [Differentiable Direct Volume Rendering](https://mlsanity.com/paper/2107.12672/) (2021)
- [Learning Adaptive Sampling and Reconstruction for Volume Visualization](https://mlsanity.com/paper/2007.10093/) (2020)
- [ScalarFlow: A Large-Scale Volumetric Data Set of Real-world Scalar Transport Flows for Computer Animation and Machine Learning](https://mlsanity.com/paper/2011.10284/) (2020)

## Built on by (most cited first, 30 of 32)

- [DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization](https://mlsanity.com/paper/2204.06504/) (2022)
- [Interactive Volume Visualization via Multi-Resolution Hash Encoding based Neural Representation](https://mlsanity.com/paper/2207.11620/) (2022)
- [HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation](https://mlsanity.com/paper/2304.04188/) (2023)
- [iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splatting](https://mlsanity.com/paper/2504.17954/) (2025)
- [ECNR: Efficient Compressive Neural Representation of Time-Varying Volumetric Datasets](https://mlsanity.com/paper/2311.12831/) (2023)
- [Adaptively Placed Multi-Grid Scene Representation Networks for Large-Scale Data Visualization](https://mlsanity.com/paper/2308.02494/) (2023)
- [Meta-INR: Efficient Encoding of Volumetric Data via Meta-Learning Implicit Neural Representation](https://mlsanity.com/paper/2502.09669/) (2025)
- [StyleRF-VolVis: Style Transfer of Neural Radiance Fields for Expressive Volume Visualization](https://mlsanity.com/paper/2408.00150/) (2024)
- [Distributed Neural Representation for Reactive in situ Visualization](https://mlsanity.com/paper/2304.10516/) (2023)
- [Explorable INR: An Implicit Neural Representation for Ensemble Simulation Enabling Efficient Spatial and Parameter Exploration](https://mlsanity.com/paper/2504.00904/) (2025)
- [Regularized Multi-Decoder Ensemble for an Error-Aware Scene Representation Network](https://mlsanity.com/paper/2407.19082/) (2024)
- [GSCache: Real-Time Radiance Caching for Volume Path Tracing using 3D Gaussian Splatting](https://mlsanity.com/paper/2507.19718/) (2025)
- [Neural Fields for Interactive Visualization of Statistical Dependencies in 3D Simulation Ensembles](https://mlsanity.com/paper/2307.02203/) (2023)
- [Efficient Neural Representation of Volumetric Data using Coordinate-Based Networks](https://mlsanity.com/paper/2401.08840/) (2024)
- [Uncertainty-Aware Deep Neural Representations for Visual Analysis of Vector Field Data](https://mlsanity.com/paper/2407.16119/) (2024)
- [Uncertainty-Informed Volume Visualization using Implicit Neural Representation](https://mlsanity.com/paper/2408.06018/) (2024)
- [F-Hash: Feature-Based Hash Design for Time-Varying Volume Visualization via Multi-Resolution Tesseract Encoding](https://mlsanity.com/paper/2507.03836/) (2025)
- [Photon Field Networks for Dynamic Real-Time Volumetric Global Illumination](https://mlsanity.com/paper/2304.07338/) (2023)
- [Improving Efficiency of Iso-Surface Extraction on Implicit Neural Representations Using Uncertainty Propagation](https://mlsanity.com/paper/2402.13861/) (2024)
- [Visual Analysis of Prediction Uncertainty in Neural Networks for Deep Image Synthesis](https://mlsanity.com/paper/2406.18545/) (2024)
- [ENTIRE: Learning-based Volume Rendering Time Prediction](https://mlsanity.com/paper/2501.12119/) (2025)
- [Efficient Compression of Structured and Unstructured Volumes via Learned 3D Gaussian Representation](https://mlsanity.com/paper/2607.01164/) (2026)
- [Monocular Depth Decomposition of Semi-Transparent Volume Renderings](https://mlsanity.com/paper/2206.13282/) (2022)
- [REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization](https://mlsanity.com/paper/2601.17689/) (2026)
- [Fixed-Budget Gaussian Volume Encoding with Structure-Aware Allocation](https://mlsanity.com/paper/2608.14112/) (2026)
- [Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes](https://mlsanity.com/paper/2501.05226/) (2025)
- [Compressive Modeling and Visualization of Multivariate Scientific Data using Implicit Neural Representation](https://mlsanity.com/paper/2510.15535/) (2025)
- [Refine Now, Query Fast: A Decoupled Refinement Paradigm for Implicit Neural Fields](https://mlsanity.com/paper/2602.15155/) (2026)
- [Lossless-INR: Lossless Volumetric Implicit Neural Representations](https://mlsanity.com/paper/2607.18150/) (2026)
- [EVOLVE: Efficient Learned Volume Compression with Variable-Rate Encoding on a Cross-Domain Database](https://mlsanity.com/paper/2607.18187/) (2026)

## Abstract (arXiv)

Despite the potential of neural scene representations to effectively compress 3D scalar fields at high reconstruction quality, the computational complexity of the training and data reconstruction step using scene representation networks limits their use in practical applications. In this paper, we analyze whether scene representation networks can be modified to reduce these limitations and whether such architectures can also be used for temporal reconstruction tasks. We propose a novel design of scene representation networks using GPU tensor cores to integrate the reconstruction seamlessly into on-chip raytracing kernels, and compare the quality and performance of this network to alternative network- and non-network-based compression schemes. The results indicate competitive quality of our design at high compression rates, and significantly faster decoding times and lower memory consumption during data reconstruction. We investigate how density gradients can be computed using the network and show an extension where density, gradient and curvature are predicted jointly. As an alternative to spatial super-resolution approaches for time-varying fields, we propose a solution that builds upon latent-space interpolation to enable random access reconstruction at arbitrary granularity. We summarize our findings in the form of an assessment of the strengths and limitations of scene representation networks changedfor compression domain volume rendering, and outline future research directions.