# Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition

arXiv: https://arxiv.org/abs/2410.20716 · HTML: https://mlsanity.com/paper/2410.20716/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 68 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [PS-FCN: A Flexible Learning Framework for Photometric Stereo](https://mlsanity.com/paper/1807.08696/) (2018)
- [Self-calibrating Deep Photometric Stereo Networks](https://mlsanity.com/paper/1903.07366/) (2019)
- [CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces](https://mlsanity.com/paper/1808.10093/) (2018)
- [Neural Inverse Rendering for General Reflectance Photometric Stereo](https://mlsanity.com/paper/1802.10328/) (2018)
- [Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces](https://mlsanity.com/paper/2012.06777/) (2020)
- [Scalable, Detailed and Mask-Free Universal Photometric Stereo](https://mlsanity.com/paper/2303.15724/) (2023)
- [Neural Reflectance for Shape Recovery with Shadow Handling](https://mlsanity.com/paper/2203.12909/) (2022)
- [SPLINE-Net: Sparse Photometric Stereo through Lighting Interpolation and Normal Estimation Networks](https://mlsanity.com/paper/1905.04088/) (2019)
- [PX-NET: Simple and Efficient Pixel-Wise Training of Photometric Stereo Networks](https://mlsanity.com/paper/2008.04933/) (2020)
- [Universal Photometric Stereo Network using Global Lighting Contexts](https://mlsanity.com/paper/2206.02452/) (2022)
- [Self-calibrating Photometric Stereo by Neural Inverse Rendering](https://mlsanity.com/paper/2207.07815/) (2022)
- [PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism](https://mlsanity.com/paper/2211.11386/) (2022)
- [Single-image RGB Photometric Stereo With Spatially-varying Albedo](https://mlsanity.com/paper/1609.04079/) (2016)
- [DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images](https://mlsanity.com/paper/2207.02025/) (2022)

## Built on by (most cited first, 4 of 4)

- [Geometry Meets Light: Leveraging Geometric Priors for Universal Photometric Stereo under Limited Multi-Illumination Cues](https://mlsanity.com/paper/2511.13015/) (2025)
- [PS-EIP: Robust Photometric Stereo Based on Event Interval Profile](https://mlsanity.com/paper/2503.18341/) (2025)
- [Near-Light Color Photometric Stereo for Mono-Chromatic Non-Lambertian Surfaces](https://mlsanity.com/paper/2601.12666/) (2026)
- [Snapshot Polarimetric Display Inverse Rendering](https://mlsanity.com/paper/2605.24915/) (2026)

## Abstract (arXiv)

In this paper, we present a groundbreaking spectrally multiplexed photometric stereo approach for recovering surface normals of dynamic surfaces without the need for calibrated lighting or sensors, a notable advancement in the field traditionally hindered by stringent prerequisites and spectral ambiguity. By embracing spectral ambiguity as an advantage, our technique enables the generation of training data without specialized multispectral rendering frameworks. We introduce a unique, physics-free network architecture, SpectraM-PS, that effectively processes multiplexed images to determine surface normals across a wide range of conditions and material types, without relying on specific physically-based knowledge. Additionally, we establish the first benchmark dataset, SpectraM14, for spectrally multiplexed photometric stereo, facilitating comprehensive evaluations against existing calibrated methods. Our contributions significantly enhance the capabilities for dynamic surface recovery, particularly in uncalibrated setups, marking a pivotal step forward in the application of photometric stereo across various domains.