# Polarimetric Multi-View Inverse Rendering

arXiv: https://arxiv.org/abs/2212.12721 · HTML: https://mlsanity.com/paper/2212.12721/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 69 references that are in the map.
- First posted on arXiv 24 Dec 2022.

## Builds on (most cited first, 9 of 9)

- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation](https://mlsanity.com/paper/1803.10409/) (2018)
- [Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials](https://mlsanity.com/paper/2001.06659/) (2020)
- [Depth from a polarisation + RGB stereo pair](https://mlsanity.com/paper/1903.12061/) (2019)
- [Monochrome and Color Polarization Demosaicking Using Edge-Aware Residual Interpolation](https://mlsanity.com/paper/2007.14292/) (2020)
- [Variational Uncalibrated Photometric Stereo under General Lighting](https://mlsanity.com/paper/1904.03942/) (2019)
- [From Shading to Local Shape](https://mlsanity.com/paper/1310.2916/) (2013)
- [Linear Differential Constraints for Photo-polarimetric Height Estimation](https://mlsanity.com/paper/1708.07718/) (2017)
- [Spectral MVIR: Joint Reconstruction of 3D Shape and Spectral Reflectance](https://mlsanity.com/paper/2104.07308/) (2021)

## Abstract (arXiv)

A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) and the degree of polarization (DoP) of reflected light are related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric rendering errors and polarimetric errors using multi-view RGB, AoP, and DoP images, where we propose a novel polarimetric cost function that enables an effective constraint on the estimated surface normal of each vertex, while considering four possible ambiguous azimuth angles revealed from the AoP measurement. The weight for the polarimetric cost is effectively determined based on the DoP measurement, which is regarded as the reliability of polarimetric information. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific surface material and lighting condition.