mlsanityData to 14 Oct 2026

Neural apparent BRDF fields for multiview photometric stereo

arXiv 2207.06793 ↗First posted 14 Jul 2022cs.CV cs.GRcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose to tackle the multiview photometric stereo problem using an extension of Neural Radiance Fields (NeRFs), conditioned on light source direction. The geometric part of our neural representation predicts surface normal direction, allowing us to reason about local surface reflectance. The appearance part of our neural representation is decomposed into a neural bidirectional reflectance function (BRDF), learnt as part of the fitting process, and a shadow prediction network (conditioned on light source direction) allowing us to model the apparent BRDF. This balance of learnt components with inductive biases based on physical image formation models allows us to extrapolate far from the light source and viewer directions observed during training. We demonstrate our approach on a multiview photometric stereo benchmark and show that competitive performance can be obtained with the neural density representation of a NeRF.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 14 Jul 2022.

Builds on

30 of 42 references.

References · 30

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  3. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  4. 2018On the Spectral Bias of Neural Networkscited by 1,341
  5. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  6. 2020Neural Sparse Voxel Fieldscited by 959
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  9. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  10. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  11. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  12. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
18 more
  1. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
  2. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  3. 2020Local Implicit Grid Representations for 3D Scenescited by 373
  4. 2021NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Imagescited by 336
  5. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  6. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  7. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  8. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  9. 2020AutoInt: Automatic Integration for Fast Neural Volume Renderingcited by 204
  10. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  11. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  12. 2020DeRF: Decomposed Radiance Fieldscited by 163
  13. 2017Optimizing the Latent Space of Generative Networkscited by 157
  14. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  15. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  16. 2019Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networkscited by 106
  17. 2020Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materialscited by 56
  18. 2019Learning a Neural 3D Texture Space from 2D Exemplarscited by 53

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 2.

5 papers.

  1. 2023Neural Microfacet Fields for Inverse Renderingcited by 38
  2. 2023MVPSNet: Fast Generalizable Multi-view Photometric Stereocited by 18
  3. 2023RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstructioncited by 13
  4. 2024Sparse Views, Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereocited by 6
  5. 2024Relighting Scenes with Object Insertions in Neural Radiance Fieldscited by 5