# MVPSNet: Fast Generalizable Multi-view Photometric Stereo

arXiv: https://arxiv.org/abs/2305.11167 · HTML: https://mlsanity.com/paper/2305.11167/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 73 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers](https://mlsanity.com/paper/2111.14600/) (2021)
- [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017)
- [PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo](https://mlsanity.com/paper/2207.11406/) (2022)
- [Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume](https://mlsanity.com/paper/1912.11746/) (2019)
- [Two-shot Spatially-varying BRDF and Shape Estimation](https://mlsanity.com/paper/2004.00403/) (2020)
- [Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials](https://mlsanity.com/paper/2001.06659/) (2020)
- [PS-FCN: A Flexible Learning Framework for Photometric Stereo](https://mlsanity.com/paper/1807.08696/) (2018)
- [Self-calibrating Deep Photometric Stereo Networks](https://mlsanity.com/paper/1903.07366/) (2019)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [Uncertainty-Aware Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2202.13071/) (2022)
- [SilNet : Single- and Multi-View Reconstruction by Learning from Silhouettes](https://mlsanity.com/paper/1711.07888/) (2017)
- [Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2110.05594/) (2021)
- [Deep Photometric Stereo for Non-Lambertian Surfaces](https://mlsanity.com/paper/2007.13145/) (2020)
- [Curvature-guided dynamic scale networks for Multi-view Stereo](https://mlsanity.com/paper/2112.05999/) (2021)
- [Multi-View Photometric Stereo Revisited](https://mlsanity.com/paper/2210.07670/) (2022)

## Built on by (most cited first, 18 of 18)

- [SuperNormal: Neural Surface Reconstruction via Multi-View Normal Integration](https://mlsanity.com/paper/2312.04803/) (2023)
- [Towards 3D Vision with Low-Cost Single-Photon Cameras](https://mlsanity.com/paper/2403.17801/) (2024)
- [MonoInstance: Enhancing Monocular Priors via Multi-view Instance Alignment for Neural Rendering and Reconstruction](https://mlsanity.com/paper/2503.18363/) (2025)
- [Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos](https://mlsanity.com/paper/2403.17915/) (2024)
- [RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction](https://mlsanity.com/paper/2312.01215/) (2023)
- [Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMs](https://mlsanity.com/paper/2406.01943/) (2024)
- [Joint Depth Prediction and Semantic Segmentation with Multi-View SAM](https://mlsanity.com/paper/2311.00134/) (2023)
- [Differentiable Point-based Inverse Rendering](https://mlsanity.com/paper/2312.02480/) (2023)
- [Sparse Views, Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo](https://mlsanity.com/paper/2404.00098/) (2024)
- [Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses](https://mlsanity.com/paper/2603.28896/) (2026)
- [Multi-view Surface Reconstruction Using Normal and Reflectance Cues](https://mlsanity.com/paper/2506.04115/) (2025)
- [NPLMV-PS: Neural Point-Light Multi-View Photometric Stereo](https://mlsanity.com/paper/2405.12057/) (2024)
- [PRM: Photometric Stereo based Large Reconstruction Model](https://mlsanity.com/paper/2412.07371/) (2024)
- [GaNI: Global and Near Field Illumination Aware Neural Inverse Rendering](https://mlsanity.com/paper/2403.15651/) (2024)
- [LUCES-MV: A Multi-View Dataset for Near-Field Point Light Source Photometric Stereo](https://mlsanity.com/paper/2412.16737/) (2024)
- [Mobile Robotic Multi-View Photometric Stereo](https://mlsanity.com/paper/2502.10842/) (2025)
- [GLOW: Global Illumination-Aware Inverse Rendering of Indoor Scenes Captured with Dynamic Co-Located Light & Camera](https://mlsanity.com/paper/2511.22857/) (2025)
- [EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups](https://mlsanity.com/paper/2602.24065/) (2026)

## Abstract (arXiv)

We propose a fast and generalizable solution to Multi-view Photometric Stereo (MVPS), called MVPSNet. The key to our approach is a feature extraction network that effectively combines images from the same view captured under multiple lighting conditions to extract geometric features from shading cues for stereo matching. We demonstrate these features, termed `Light Aggregated Feature Maps' (LAFM), are effective for feature matching even in textureless regions, where traditional multi-view stereo methods fail. Our method produces similar reconstruction results to PS-NeRF, a state-of-the-art MVPS method that optimizes a neural network per-scene, while being 411× faster (105 seconds vs. 12 hours) in inference. Additionally, we introduce a new synthetic dataset for MVPS, sMVPS, which is shown to be effective to train a generalizable MVPS method.