# Fast Image-based Neural Relighting with Translucency-Reflection Modeling

arXiv: https://arxiv.org/abs/2306.09322 · HTML: https://mlsanity.com/paper/2306.09322/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 20 of 20)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images](https://mlsanity.com/paper/2007.09892/) (2020)
- [Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering](https://mlsanity.com/paper/2103.15208/) (2021)
- [Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination](https://mlsanity.com/paper/2207.13607/) (2022)
- [RelightableHands: Efficient Neural Relighting of Articulated Hand Models](https://mlsanity.com/paper/2302.04866/) (2023)
- [Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Composition](https://mlsanity.com/paper/2303.06138/) (2023)
- [Neural Relightable Participating Media Rendering](https://mlsanity.com/paper/2110.12993/) (2021)

## Built on by (most cited first, 6 of 6)

- [BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2408.13370/) (2024)
- [Artist-Friendly Relightable and Animatable Neural Heads](https://mlsanity.com/paper/2312.03420/) (2023)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [Subsurface Scattering for 3D Gaussian Splatting](https://mlsanity.com/paper/2408.12282/) (2024)
- [Neural Acquisition & Representation of Subsurface Scattering](https://mlsanity.com/paper/2606.02292/) (2026)
- [DIAMOND-SSS: Diffusion-Augmented Multi-View Optimization for Data-efficient SubSurface Scattering](https://mlsanity.com/paper/2601.12020/) (2026)

## Abstract (arXiv)

Image-based lighting (IBL) is a widely used technique that renders objects using a high dynamic range image or environment map. However, aggregating the irradiance at the object's surface is computationally expensive, in particular for non-opaque, translucent materials that require volumetric rendering techniques. In this paper we present a fast neural 3D reconstruction and relighting model that extends volumetric implicit models such as neural radiance fields to be relightable using IBL. It is general enough to handle materials that exhibit complex light transport effects, such as translucency and glossy reflections from detailed surface geometry, producing realistic and compelling results. Rendering can be within a second at 800×800 resolution (0.72s on an NVIDIA 3090 GPU and 0.30s on an A100 GPU) without engineering optimization. Our code and dataset are available at https://zhusz.github.io/TRHM-Webpage/.