# Universal Photometric Stereo Network using Global Lighting Contexts

arXiv: https://arxiv.org/abs/2206.02452 · HTML: https://mlsanity.com/paper/2206.02452/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 6 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding](https://mlsanity.com/paper/2103.15358/) (2021)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials](https://mlsanity.com/paper/2001.06659/) (2020)
- [PS-FCN: A Flexible Learning Framework for Photometric Stereo](https://mlsanity.com/paper/1807.08696/) (2018)
- [Self-calibrating Deep Photometric Stereo Networks](https://mlsanity.com/paper/1903.07366/) (2019)
- [CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces](https://mlsanity.com/paper/1808.10093/) (2018)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [Neural Inverse Rendering for General Reflectance Photometric Stereo](https://mlsanity.com/paper/1802.10328/) (2018)
- [Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces](https://mlsanity.com/paper/2012.06777/) (2020)
- [PX-NET: Simple and Efficient Pixel-Wise Training of Photometric Stereo Networks](https://mlsanity.com/paper/2008.04933/) (2020)
- [Variational Uncalibrated Photometric Stereo under General Lighting](https://mlsanity.com/paper/1904.03942/) (2019)
- [On the well-posedness of uncalibrated photometric stereo under general lighting](https://mlsanity.com/paper/1911.07268/) (2019)
- [Single Day Outdoor Photometric Stereo](https://mlsanity.com/paper/1803.10850/) (2018)

## Built on by (most cited first, 18 of 18)

- [Scalable, Detailed and Mask-Free Universal Photometric Stereo](https://mlsanity.com/paper/2303.15724/) (2023)
- [Deep Learning Methods for Calibrated Photometric Stereo and Beyond](https://mlsanity.com/paper/2212.08414/) (2022)
- [Light of Normals: Unified Feature Representation for Universal Photometric Stereo](https://mlsanity.com/paper/2506.18882/) (2025)
- [Multistable Shape from Shading Emerges from Patch Diffusion](https://mlsanity.com/paper/2405.14530/) (2024)
- [Multi-view Surface Reconstruction Using Normal and Reflectance Cues](https://mlsanity.com/paper/2506.04115/) (2025)
- [Differentiable Display Photometric Stereo](https://mlsanity.com/paper/2306.13325/) (2023)
- [NPLMV-PS: Neural Point-Light Multi-View Photometric Stereo](https://mlsanity.com/paper/2405.12057/) (2024)
- [Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition](https://mlsanity.com/paper/2410.20716/) (2024)
- [PRM: Photometric Stereo based Large Reconstruction Model](https://mlsanity.com/paper/2412.07371/) (2024)
- [A Real-world Display Inverse Rendering Dataset](https://mlsanity.com/paper/2508.14411/) (2025)
- [MS-PS: A Multi-Scale Network for Photometric Stereo With a New Comprehensive Training Dataset](https://mlsanity.com/paper/2211.14118/) (2022)
- [LUCES-MV: A Multi-View Dataset for Near-Field Point Light Source Photometric Stereo](https://mlsanity.com/paper/2412.16737/) (2024)
- [Monocular Normal Estimation via Shading Sequence Estimation](https://mlsanity.com/paper/2602.09929/) (2026)
- [Spin-UP: Spin Light for Natural Light Uncalibrated Photometric Stereo](https://mlsanity.com/paper/2404.01612/) (2024)
- [PS-EIP: Robust Photometric Stereo Based on Event Interval Profile](https://mlsanity.com/paper/2503.18341/) (2025)
- [Near-light Photometric Stereo with Symmetric Lights](https://mlsanity.com/paper/2603.16404/) (2026)
- [FlashNormal: Detailed Surface Normal Estimation from Flash and No-Flash Images](https://mlsanity.com/paper/2608.25360/) (2026)
- [Calibration-Free Surface Normals Estimation in Vision-Based Tactile Sensing using Universal Photometric Stereo](https://mlsanity.com/paper/2609.31754/) (2026)

## Abstract (arXiv)

This paper tackles a new photometric stereo task, named universal photometric stereo. Unlike existing tasks that assumed specific physical lighting models; hence, drastically limited their usability, a solution algorithm of this task is supposed to work for objects with diverse shapes and materials under arbitrary lighting variations without assuming any specific models. To solve this extremely challenging task, we present a purely data-driven method, which eliminates the prior assumption of lighting by replacing the recovery of physical lighting parameters with the extraction of the generic lighting representation, named global lighting contexts. We use them like lighting parameters in a calibrated photometric stereo network to recover surface normal vectors pixelwisely. To adapt our network to a wide variety of shapes, materials and lightings, it is trained on a new synthetic dataset which simulates the appearance of objects in the wild. Our method is compared with other state-of-the-art uncalibrated photometric stereo methods on our test data to demonstrate the significance of our method.