# Deep Appearance Prefiltering

arXiv: https://arxiv.org/abs/2211.05932 · HTML: https://mlsanity.com/paper/2211.05932/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 116 references that are in the map.
- First posted on arXiv 8 Nov 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Deep Surface Light Fields](https://mlsanity.com/paper/1810.06514/) (2018)
- [Light Field Video Capture Using a Learning-Based Hybrid Imaging System](https://mlsanity.com/paper/1705.02997/) (2017)
- [Deep Appearance Maps](https://mlsanity.com/paper/1804.00863/) (2018)
- [A Radiative Transfer Framework for Spatially-Correlated Materials](https://mlsanity.com/paper/1805.02651/) (2018)
- [Gaussian Material Synthesis](https://mlsanity.com/paper/1804.08369/) (2018)
- [An unbiased ray-marching transmittance estimator](https://mlsanity.com/paper/2102.10294/) (2021)
- [Neural View-Interpolation for Sparse Light Field Video](https://mlsanity.com/paper/1910.13921/) (2019)

## Built on by (most cited first, 4 of 4)

- [Neural Free-Viewpoint Relighting for Glossy Indirect Illumination](https://mlsanity.com/paper/2307.06335/) (2023)
- [NeuBTF: Neural fields for BTF encoding and transfer](https://mlsanity.com/paper/2307.01199/) (2023)
- [Efficient Scene Appearance Aggregation for Level-of-Detail Rendering](https://mlsanity.com/paper/2409.03761/) (2024)
- [Gabor Fields: Orientation-Selective Level-of-Detail for Volume Rendering](https://mlsanity.com/paper/2602.05081/) (2026)

## Abstract (arXiv)

Physically based rendering of complex scenes can be prohibitively costly with a potentially unbounded and uneven distribution of complexity across the rendered image. The goal of an ideal level of detail (LoD) method is to make rendering costs independent of the 3D scene complexity, while preserving the appearance of the scene. However, current prefiltering LoD methods are limited in the appearances they can support due to their reliance of approximate models and other heuristics. We propose the first comprehensive multi-scale LoD framework for prefiltering 3D environments with complex geometry and materials (e.g., the Disney BRDF), while maintaining the appearance with respect to the ray-traced reference. Using a multi-scale hierarchy of the scene, we perform a data-driven prefiltering step to obtain an appearance phase function and directional coverage mask at each scale. At the heart of our approach is a novel neural representation that encodes this information into a compact latent form that is easy to decode inside a physically based renderer. Once a scene is baked out, our method requires no original geometry, materials, or textures at render time. We demonstrate that our approach compares favorably to state-of-the-art prefiltering methods and achieves considerable savings in memory for complex scenes.