# Real-Time Neural Appearance Models

arXiv: https://arxiv.org/abs/2305.02678 · HTML: https://mlsanity.com/paper/2305.02678/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2023.

## Built on by (most cited first, 21 of 21)

- [Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling](https://mlsanity.com/paper/2405.14847/) (2024)
- [Generative World Renderer](https://mlsanity.com/paper/2604.02329/) (2026)
- [Real-Time Neural BRDF with Spherically Distributed Primitives](https://mlsanity.com/paper/2310.08332/) (2023)
- [Real-Time Neural Materials using Block-Compressed Features](https://mlsanity.com/paper/2311.16121/) (2023)
- [A Hierarchical Architecture for Neural Materials](https://mlsanity.com/paper/2307.10135/) (2023)
- [Neural Product Importance Sampling via Warp Composition](https://mlsanity.com/paper/2409.18974/) (2024)
- [Neural BRDF Importance Sampling by Reparameterization](https://mlsanity.com/paper/2505.08998/) (2025)
- [RNA: Relightable Neural Assets](https://mlsanity.com/paper/2312.09398/) (2023)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [SYNOSIS: Image synthesis pipeline for machine vision in metal surface inspection](https://mlsanity.com/paper/2410.14844/) (2024)
- [Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-realistic Rendering](https://mlsanity.com/paper/2504.12273/) (2025)
- [VideoNeuMat: Neural Material Extraction from Generative Video Models](https://mlsanity.com/paper/2602.07272/) (2026)
- [8DNA: 8D Neural Asset Light Transport by Distribution Learning](https://mlsanity.com/paper/2604.25129/) (2026)
- [Real-time Neural Woven Fabric Rendering](https://mlsanity.com/paper/2406.17782/) (2024)
- [From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models](https://mlsanity.com/paper/2409.16024/) (2024)
- [Neural Two-Level Monte Carlo Real-Time Rendering](https://mlsanity.com/paper/2412.04634/) (2024)
- [GratNet: A Photorealistic Neural Shader for Diffractive Surfaces](https://mlsanity.com/paper/2506.15815/) (2025)
- [Rendering-Aware Sparse Sampling for BRDF Acquisition](https://mlsanity.com/paper/2604.26740/) (2026)
- [Extracting Neural Materials from Images](https://mlsanity.com/paper/2606.26715/) (2026)
- [A Hybrid Neural-Microfacet BRDF Model for Real-Time Rendering](https://mlsanity.com/paper/2608.09604/) (2026)
- [Compact Neural Appearance Models for Efficient Gaussian Splatting](https://mlsanity.com/paper/2609.05255/) (2026)

## Abstract (arXiv)

We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior -- transformation of directions into learned shading frames -- facilitates accurate reconstruction of mesoscale effects. The second prior -- a microfacet sampling distribution -- allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.