# Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-realistic Rendering

arXiv: https://arxiv.org/abs/2504.12273 · HTML: https://mlsanity.com/paper/2504.12273/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 16 Apr 2025.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Splatter Image: Ultra-Fast Single-View 3D Reconstruction](https://mlsanity.com/paper/2312.13150/) (2023)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [NeRFReN: Neural Radiance Fields with Reflections](https://mlsanity.com/paper/2111.15234/) (2021)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting](https://mlsanity.com/paper/2402.18848/) (2024)
- [3D Gaussian Splatting with Deferred Reflection](https://mlsanity.com/paper/2404.18454/) (2024)
- [Multi-View Mesh Reconstruction with Neural Deferred Shading](https://mlsanity.com/paper/2212.04386/) (2022)
- [Neural Radiosity](https://mlsanity.com/paper/2105.12319/) (2021)
- [MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors](https://mlsanity.com/paper/2409.15273/) (2024)
- [Real-Time Neural Appearance Models](https://mlsanity.com/paper/2305.02678/) (2023)

## Built on by (most cited first, 2 of 2)

- [A Convolutional Neural Deferred Shader for Physics Based Rendering](https://mlsanity.com/paper/2512.19522/) (2025)
- [Deepfakes and Synthetic Media: Generation, Detection, and Governance](https://mlsanity.com/paper/2609.25017/) (2026)

## Abstract (arXiv)

Deep learning based rendering has achieved major improvements in photo-realistic image synthesis, with potential applications including visual effects in movies and photo-realistic scene building in video games. However, a significant limitation is the difficulty of decomposing the illumination and material parameters, which limits such methods to reconstructing an input scene, without any possibility to control these parameters. This paper introduces a novel physics based neural deferred shading pipeline to decompose the data-driven rendering process, learn a generalizable shading function to produce photo-realistic results for shading and relighting tasks; we also propose a shadow estimator to efficiently mimic shadowing effects. Our model achieves improved performance compared to classical models and a state-of-art neural shading model, and enables generalizable photo-realistic shading from arbitrary illumination input.