# Real-time Neural Radiance Caching for Path Tracing

arXiv: https://arxiv.org/abs/2106.12372 · HTML: https://mlsanity.com/paper/2106.12372/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 87 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [Neural Importance Sampling](https://mlsanity.com/paper/1808.03856/) (2018)
- [Deep Shading: Convolutional Neural Networks for Screen-Space Shading](https://mlsanity.com/paper/1603.06078/) (2016)
- [Neural Control Variates](https://mlsanity.com/paper/2006.01524/) (2020)
- [Scalable Second Order Optimization for Deep Learning](https://mlsanity.com/paper/2002.09018/) (2020)
- [Learning Light Transport the Reinforced Way](https://mlsanity.com/paper/1701.07403/) (2017)
- [Deep-learning the Latent Space of Light Transport](https://mlsanity.com/paper/1811.04756/) (2018)
- [Online path sampling control with progressive spatio-temporal filtering](https://mlsanity.com/paper/2005.07547/) (2020)
- [Integral Equations and Machine Learning](https://mlsanity.com/paper/1712.06115/) (2017)
- [Second-Order Occlusion-Aware Volumetric Radiance Caching](https://mlsanity.com/paper/1805.09258/) (2018)

## Built on by (most cited first, 30 of 91)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2311.13384/) (2023)
- [3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2312.09228/) (2023)
- [Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images](https://mlsanity.com/paper/2311.13398/) (2023)
- [3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos](https://mlsanity.com/paper/2403.01444/) (2024)
- [Variable Bitrate Neural Fields](https://mlsanity.com/paper/2206.07707/) (2022)
- [DiffusionPDE: Generative PDE-Solving Under Partial Observation](https://mlsanity.com/paper/2406.17763/) (2024)
- [3D Gaussian as a New Era: A Survey](https://mlsanity.com/paper/2402.07181/) (2024)
- [Temporal Interpolation Is All You Need for Dynamic Neural Radiance Fields](https://mlsanity.com/paper/2302.09311/) (2023)
- [DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models](https://mlsanity.com/paper/2501.18590/) (2025)
- [Spatiotemporal implicit neural representation for unsupervised dynamic MRI reconstruction](https://mlsanity.com/paper/2301.00127/) (2022)
- [Fast Neural Representations for Direct Volume Rendering](https://mlsanity.com/paper/2112.01579/) (2021)
- [SceneDreamer360: Text-Driven 3D-Consistent Scene Generation with Panoramic Gaussian Splatting](https://mlsanity.com/paper/2408.13711/) (2024)
- [Random-Access Neural Compression of Material Textures](https://mlsanity.com/paper/2305.17105/) (2023)
- [RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination](https://mlsanity.com/paper/2505.21925/) (2025)
- [Robust Camera Pose Refinement for Multi-Resolution Hash Encoding](https://mlsanity.com/paper/2302.01571/) (2023)
- [Efficient physics-informed neural networks using hash encoding](https://mlsanity.com/paper/2302.13397/) (2023)
- [Dynamic Mesh-Aware Radiance Fields](https://mlsanity.com/paper/2309.04581/) (2023)
- [NeuralVDB: High-resolution Sparse Volume Representation using Hierarchical Neural Networks](https://mlsanity.com/paper/2208.04448/) (2022)
- [Interactive Volume Visualization via Multi-Resolution Hash Encoding based Neural Representation](https://mlsanity.com/paper/2207.11620/) (2022)
- [HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation](https://mlsanity.com/paper/2304.04188/) (2023)
- [Compact Neural Graphics Primitives with Learned Hash Probing](https://mlsanity.com/paper/2312.17241/) (2023)
- [Efficient Image Generation with Variadic Attention Heads](https://mlsanity.com/paper/2211.05770/) (2022)
- [N-Dimensional Gaussians for Fitting of High Dimensional Functions](https://mlsanity.com/paper/2405.20067/) (2024)
- [Neural Spline Fields for Burst Image Fusion and Layer Separation](https://mlsanity.com/paper/2312.14235/) (2023)
- [Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling](https://mlsanity.com/paper/2405.14847/) (2024)
- [Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering](https://mlsanity.com/paper/2409.05867/) (2024)
- [Active Exploration for Neural Global Illumination of Variable Scenes](https://mlsanity.com/paper/2203.08272/) (2022)
- [MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields](https://mlsanity.com/paper/2212.08328/) (2022)

## Abstract (arXiv)

We present a real-time neural radiance caching method for path-traced global illumination. Our system is designed to handle fully dynamic scenes, and makes no assumptions about the lighting, geometry, and materials. The data-driven nature of our approach sidesteps many difficulties of caching algorithms, such as locating, interpolating, and updating cache points. Since pretraining neural networks to handle novel, dynamic scenes is a formidable generalization challenge, we do away with pretraining and instead achieve generalization via adaptation, i.e. we opt for training the radiance cache while rendering. We employ self-training to provide low-noise training targets and simulate infinite-bounce transport by merely iterating few-bounce training updates. The updates and cache queries incur a mild overhead -- about 2.6ms on full HD resolution -- thanks to a streaming implementation of the neural network that fully exploits modern hardware. We demonstrate significant noise reduction at the cost of little induced bias, and report state-of-the-art, real-time performance on a number of challenging scenarios.