# Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo

arXiv: https://arxiv.org/abs/2103.12106 · HTML: https://mlsanity.com/paper/2103.12106/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)
- [PS-FCN: A Flexible Learning Framework for Photometric Stereo](https://mlsanity.com/paper/1807.08696/) (2018)
- [CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces](https://mlsanity.com/paper/1808.10093/) (2018)
- [Neural Inverse Rendering for General Reflectance Photometric Stereo](https://mlsanity.com/paper/1802.10328/) (2018)
- [SPLINE-Net: Sparse Photometric Stereo through Lighting Interpolation and Normal Estimation Networks](https://mlsanity.com/paper/1905.04088/) (2019)
- [Deep Photometric Stereo for Non-Lambertian Surfaces](https://mlsanity.com/paper/2007.13145/) (2020)
- [PX-NET: Simple and Efficient Pixel-Wise Training of Photometric Stereo Networks](https://mlsanity.com/paper/2008.04933/) (2020)

## Built on by (most cited first, 4 of 4)

- [Neural Reflectance for Shape Recovery with Shadow Handling](https://mlsanity.com/paper/2203.12909/) (2022)
- [Self-calibrating Photometric Stereo by Neural Inverse Rendering](https://mlsanity.com/paper/2207.07815/) (2022)
- [Deep Learning Methods for Calibrated Photometric Stereo and Beyond](https://mlsanity.com/paper/2212.08414/) (2022)
- [MS-PS: A Multi-Scale Network for Photometric Stereo With a New Comprehensive Training Dataset](https://mlsanity.com/paper/2211.14118/) (2022)

## Abstract (arXiv)

The problem of estimating a surface shape from its observed reflectance properties still remains a challenging task in computer vision. The presence of global illumination effects such as inter-reflections or cast shadows makes the task particularly difficult for non-convex real-world surfaces. State-of-the-art methods for calibrated photometric stereo address these issues using convolutional neural networks (CNNs) that primarily aim to capture either the spatial context among adjacent pixels or the photometric one formed by illuminating a sample from adjacent directions. In this paper, we bridge these two objectives and introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. In contrast to existing approaches that rely on standard 2D CNNs and regress directly to surface normals, we argue that using separable 4D convolutions and regressing to 2D Gaussian heat-maps severely reduces the size of the network and makes inference more efficient. Our experimental results on a real-world photometric stereo benchmark show that the proposed approach outperforms the existing methods both in efficiency and accuracy.