# CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces

arXiv: https://arxiv.org/abs/1808.10093 · HTML: https://mlsanity.com/paper/1808.10093/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2018.

## Built on by (most cited first, 30 of 45)

- [PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo](https://mlsanity.com/paper/2207.11406/) (2022)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [Uncertainty-Aware Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2202.13071/) (2022)
- [S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint](https://mlsanity.com/paper/2210.08936/) (2022)
- [Scalable, Detailed and Mask-Free Universal Photometric Stereo](https://mlsanity.com/paper/2303.15724/) (2023)
- [Neural Reflectance for Shape Recovery with Shadow Handling](https://mlsanity.com/paper/2203.12909/) (2022)
- [Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2110.05594/) (2021)
- [Universal Photometric Stereo Network using Global Lighting Contexts](https://mlsanity.com/paper/2206.02452/) (2022)
- [Self-calibrating Photometric Stereo by Neural Inverse Rendering](https://mlsanity.com/paper/2207.07815/) (2022)
- [Multi-View Photometric Stereo Revisited](https://mlsanity.com/paper/2210.07670/) (2022)
- [LUCES: A Dataset for Near-Field Point Light Source Photometric Stereo](https://mlsanity.com/paper/2104.13135/) (2021)
- [PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism](https://mlsanity.com/paper/2211.11386/) (2022)
- [RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction](https://mlsanity.com/paper/2312.01215/) (2023)
- [Fast Light-Weight Near-Field Photometric Stereo](https://mlsanity.com/paper/2203.16515/) (2022)
- [A CNN Based Approach for the Point-Light Photometric Stereo Problem](https://mlsanity.com/paper/2210.04655/) (2022)
- [Deep Learning Methods for Calibrated Photometric Stereo and Beyond](https://mlsanity.com/paper/2212.08414/) (2022)
- [Light of Normals: Unified Feature Representation for Universal Photometric Stereo](https://mlsanity.com/paper/2506.18882/) (2025)
- [Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo](https://mlsanity.com/paper/2110.05621/) (2021)
- [DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images](https://mlsanity.com/paper/2207.02025/) (2022)
- [DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering](https://mlsanity.com/paper/2303.15101/) (2023)
- [Multi-view Surface Reconstruction Using Normal and Reflectance Cues](https://mlsanity.com/paper/2506.04115/) (2025)
- [Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo](https://mlsanity.com/paper/2103.12106/) (2021)
- [Incorporating Lambertian Priors into Surface Normals Measurement](https://mlsanity.com/paper/2107.07192/) (2021)
- [NPLMV-PS: Neural Point-Light Multi-View Photometric Stereo](https://mlsanity.com/paper/2405.12057/) (2024)
- [Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition](https://mlsanity.com/paper/2410.20716/) (2024)
- [A Real-world Display Inverse Rendering Dataset](https://mlsanity.com/paper/2508.14411/) (2025)
- [MS-PS: A Multi-Scale Network for Photometric Stereo With a New Comprehensive Training Dataset](https://mlsanity.com/paper/2211.14118/) (2022)
- [A Neural Height-Map Approach for the Binocular Photometric Stereo Problem](https://mlsanity.com/paper/2311.05958/) (2023)
- [LUCES-MV: A Multi-View Dataset for Near-Field Point Light Source Photometric Stereo](https://mlsanity.com/paper/2412.16737/) (2024)
- [Spin-UP: Spin Light for Natural Light Uncalibrated Photometric Stereo](https://mlsanity.com/paper/2404.01612/) (2024)

## Abstract (arXiv)

Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo input and surface normals of a scene. For handling unordered, arbitrary number of input images, we merge all the input data to the intermediate representation called it observation map that has a fixed shape, is able to be fed into a CNN. To improve both training and prediction, we take into account the rotational pseudo-invariance of the observation map that is derived from the isotropic constraint. For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered. Our experimental results on both synthetic and real datasets show that our method outperforms conventional BRDF-based photometric stereo algorithms especially when scenes are highly non-convex.