# Hardware Acceleration of Neural Graphics arXiv: https://arxiv.org/abs/2303.05735 ยท HTML: https://mlsanity.com/paper/2303.05735/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 17 of its 43 references that are in the map. - First posted on arXiv 10 Mar 2023. ## Builds on (most cited first, 17 of 17) - [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022) - [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019) - [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020) - [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021) - [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020) - [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018) - [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020) - [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018) - [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019) - [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020) - [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021) - [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020) - [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020) - [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](https://mlsanity.com/paper/2103.03231/) (2021) - [Real-time Neural Radiance Caching for Path Tracing](https://mlsanity.com/paper/2106.12372/) (2021) - [The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches](https://mlsanity.com/paper/1803.01164/) (2018) - [Energy-Efficient ConvNets Through Approximate Computing](https://mlsanity.com/paper/1603.06777/) (2016) ## Built on by (most cited first, 10 of 10) - [3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors](https://mlsanity.com/paper/2410.16266/) (2024) - [DISTWAR: Fast Differentiable Rendering on Raster-based Rendering Pipelines](https://mlsanity.com/paper/2401.05345/) (2023) - [CoGS: Controllable Gaussian Splatting](https://mlsanity.com/paper/2312.05664/) (2023) - [Neural Rendering and Its Hardware Acceleration: A Review](https://mlsanity.com/paper/2402.00028/) (2024) - [VR-Pipe: Streamlining Hardware Graphics Pipeline for Volume Rendering](https://mlsanity.com/paper/2502.17078/) (2025) - [Towards Energy-Efficiency by Navigating the Trilemma of Energy, Latency, and Accuracy](https://mlsanity.com/paper/2409.04018/) (2024) - [NeRFs are Mirror Detectors: Using Structural Similarity for Multi-View Mirror Scene Reconstruction with 3D Surface Primitives](https://mlsanity.com/paper/2501.04074/) (2025) - [Uni-Render: A Unified Accelerator for Real-Time Rendering Across Diverse Neural Renderers](https://mlsanity.com/paper/2503.23644/) (2025) - [GRTX: Efficient Ray Tracing for 3D Gaussian-Based Rendering](https://mlsanity.com/paper/2601.20429/) (2026) - [HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction](https://mlsanity.com/paper/2605.16873/) (2026) ## Abstract (arXiv) Rendering and inverse-rendering algorithms that drive conventional computer graphics have recently been superseded by neural representations (NR). NRs have recently been used to learn the geometric and the material properties of the scenes and use the information to synthesize photorealistic imagery, thereby promising a replacement for traditional rendering algorithms with scalable quality and predictable performance. In this work we ask the question: Does neural graphics (NG) need hardware support? We studied representative NG applications showing that, if we want to render 4k res. at 60FPS there is a gap of 1.5X-55X in the desired performance on current GPUs. For AR/VR applications, there is an even larger gap of 2-4 OOM between the desired performance and the required system power. We identify that the input encoding and the MLP kernels are the performance bottlenecks, consuming 72%,60% and 59% of application time for multi res. hashgrid, multi res. densegrid and low res. densegrid encodings, respectively. We propose a NG processing cluster, a scalable and flexible hardware architecture that directly accelerates the input encoding and MLP kernels through dedicated engines and supports a wide range of NG applications. We also accelerate the rest of the kernels by fusing them together in Vulkan, which leads to 9.94X kernel-level performance improvement compared to un-fused implementation of the pre-processing and the post-processing kernels. Our results show that, NGPC gives up to 58X end-to-end application-level performance improvement, for multi res. hashgrid encoding on average across the four NG applications, the performance benefits are 12X,20X,33X and 39X for the scaling factor of 8,16,32 and 64, respectively. Our results show that with multi res. hashgrid encoding, NGPC enables the rendering of 4k res. at 30FPS for NeRF and 8k res. at 120FPS for all our other NG applications.