# NeMF: Inverse Volume Rendering with Neural Microflake Field

arXiv: https://arxiv.org/abs/2304.00782 · HTML: https://mlsanity.com/paper/2304.00782/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 30 of 31)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [MetaSDF: Meta-learning Signed Distance Functions](https://mlsanity.com/paper/2006.09662/) (2020)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization](https://mlsanity.com/paper/1912.07109/) (2019)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images](https://mlsanity.com/paper/2007.09892/) (2020)
- [Neural Light Transport for Relighting and View Synthesis](https://mlsanity.com/paper/2008.03806/) (2020)
- [Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images](https://mlsanity.com/paper/2003.12642/) (2020)
- [Relighting4D: Neural Relightable Human from Videos](https://mlsanity.com/paper/2207.07104/) (2022)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting](https://mlsanity.com/paper/1708.01670/) (2017)
- [A Neural Rendering Framework for Free-Viewpoint Relighting](https://mlsanity.com/paper/1911.11530/) (2019)
- [Seeing the World in a Bag of Chips](https://mlsanity.com/paper/2001.04642/) (2020)
- [Object-based Illumination Estimation with Rendering-aware Neural Networks](https://mlsanity.com/paper/2008.02514/) (2020)
- [Deep SVBRDF Estimation on Real Materials](https://mlsanity.com/paper/2010.04143/) (2020)

## Built on by (most cited first, 30 of 34)

- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [GS^3: Efficient Relighting with Triple Gaussian Splatting](https://mlsanity.com/paper/2410.11419/) (2024)
- [SpecNeRF: Gaussian Directional Encoding for Specular Reflections](https://mlsanity.com/paper/2312.13102/) (2023)
- [Ref-GS: Directional Factorization for 2D Gaussian Splatting](https://mlsanity.com/paper/2412.00905/) (2024)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [RelitLRM: Generative Relightable Radiance for Large Reconstruction Models](https://mlsanity.com/paper/2410.06231/) (2024)
- [Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing](https://mlsanity.com/paper/2308.03280/) (2023)
- [IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing](https://mlsanity.com/paper/2312.05210/) (2023)
- [Mirror-3DGS: Incorporating Mirror Reflections into 3D Gaussian Splatting](https://mlsanity.com/paper/2404.01168/) (2024)
- [RNG: Relightable Neural Gaussians](https://mlsanity.com/paper/2409.19702/) (2024)
- [LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields](https://mlsanity.com/paper/2504.20026/) (2025)
- [Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects](https://mlsanity.com/paper/2305.17929/) (2023)
- [Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Video](https://mlsanity.com/paper/2407.15212/) (2024)
- [VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization](https://mlsanity.com/paper/2310.11864/) (2023)
- [BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2408.13370/) (2024)
- [TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through Neural Radiance Fields](https://mlsanity.com/paper/2310.10650/) (2023)
- [Interactive Rendering of Relightable and Animatable Gaussian Avatars](https://mlsanity.com/paper/2407.10707/) (2024)
- [AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction](https://mlsanity.com/paper/2410.01202/) (2024)
- [Many-Worlds Inverse Rendering](https://mlsanity.com/paper/2408.16005/) (2024)
- [RNA: Relightable Neural Assets](https://mlsanity.com/paper/2312.09398/) (2023)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [NeRFs are Mirror Detectors: Using Structural Similarity for Multi-View Mirror Scene Reconstruction with 3D Surface Primitives](https://mlsanity.com/paper/2501.04074/) (2025)
- [ROSA: Reconstructing Object Shape and Appearance Textures by Adaptive Detail Transfer](https://mlsanity.com/paper/2501.18595/) (2025)
- [LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows](https://mlsanity.com/paper/2604.05182/) (2026)
- [Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction](https://mlsanity.com/paper/2407.05771/) (2024)
- [UniVoxel: Fast Inverse Rendering by Unified Voxelization of Scene Representation](https://mlsanity.com/paper/2407.19542/) (2024)
- [Material Transforms from Disentangled NeRF Representations](https://mlsanity.com/paper/2411.08037/) (2024)
- [GTSR: Subsurface Scattering Awared 3D Gaussians for Translucent Surface Reconstruction](https://mlsanity.com/paper/2603.22036/) (2026)
- [Optimized View and Geometry Distillation from Multi-view Diffuser](https://mlsanity.com/paper/2312.06198/) (2023)
- [Exploration and Improvement of Nerf-based 3D Scene Editing Techniques](https://mlsanity.com/paper/2401.12456/) (2024)

## Abstract (arXiv)

Recovering the physical attributes of an object's appearance from its images captured under an unknown illumination is challenging yet essential for photo-realistic rendering. Recent approaches adopt the emerging implicit scene representations and have shown impressive results.However, they unanimously adopt a surface-based representation,and hence can not well handle scenes with very complex geometry, translucent object and etc. In this paper, we propose to conduct inverse volume rendering, in contrast to surface-based, by representing a scene using microflake volume, which assumes the space is filled with infinite small flakes and light reflects or scatters at each spatial location according to microflake distributions. We further adopt the coordinate networks to implicitly encode the microflake volume, and develop a differentiable microflake volume renderer to train the network in an end-to-end way in principle.Our NeMF enables effective recovery of appearance attributes for highly complex geometry and scattering object, enables high-quality relighting, material editing, and especially simulates volume rendering effects, such as scattering, which is infeasible for surface-based approaches.