# Deep Polarization Imaging for 3D shape and SVBRDF Acquisition

arXiv: https://arxiv.org/abs/2105.02875 · HTML: https://mlsanity.com/paper/2105.02875/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 9 of 9)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Single-Image SVBRDF Capture with a Rendering-Aware Deep Network](https://mlsanity.com/paper/1810.09718/) (2018)
- [Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image](https://mlsanity.com/paper/1804.05790/) (2018)
- [Two-shot Spatially-varying BRDF and Shape Estimation](https://mlsanity.com/paper/2004.00403/) (2020)
- [Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks](https://mlsanity.com/paper/1809.00886/) (2018)
- [Flexible SVBRDF Capture with a Multi-Image Deep Network](https://mlsanity.com/paper/1906.11557/) (2019)
- [LIME: Live Intrinsic Material Estimation](https://mlsanity.com/paper/1801.01075/) (2018)
- [Polarimetric Multi-View Inverse Rendering](https://mlsanity.com/paper/2007.08830/) (2020)
- [Guided Fine-Tuning for Large-Scale Material Transfer](https://mlsanity.com/paper/2007.03059/) (2020)

## Built on by (most cited first, 30 of 56)

- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models](https://mlsanity.com/paper/2405.17176/) (2024)
- [MatSynth: A Modern PBR Materials Dataset](https://mlsanity.com/paper/2401.06056/) (2024)
- [Material Palette: Extraction of Materials from a Single Image](https://mlsanity.com/paper/2311.17060/) (2023)
- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [High-Res Facial Appearance Capture from Polarized Smartphone Images](https://mlsanity.com/paper/2212.01160/) (2022)
- [Transparent Shape from a Single View Polarization Image](https://mlsanity.com/paper/2204.06331/) (2022)
- [UMat: Uncertainty-Aware Single Image High Resolution Material Capture](https://mlsanity.com/paper/2305.16312/) (2023)
- [Polarization Multiplexed Diffractive Computing: All-Optical Implementation of a Group of Linear Transformations Through a Polarization-Encoded Diffractive Network](https://mlsanity.com/paper/2203.13482/) (2022)
- [Neural Spectro-polarimetric Fields](https://mlsanity.com/paper/2306.12562/) (2023)
- [Simultaneous Acquisition of High Quality RGB Image and Polarization Information using a Sparse Polarization Sensor](https://mlsanity.com/paper/2209.13106/) (2022)
- [Inverse Rendering of Translucent Objects using Physical and Neural Renderers](https://mlsanity.com/paper/2305.08336/) (2023)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [Polarimetric Inverse Rendering for Transparent Shapes Reconstruction](https://mlsanity.com/paper/2208.11836/) (2022)
- [Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset](https://mlsanity.com/paper/2311.17396/) (2023)
- [PolGS: Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstruction](https://mlsanity.com/paper/2509.19726/) (2025)
- [Sparse Ellipsometry: Portable Acquisition of Polarimetric SVBRDF and Shape with Unstructured Flash Photography](https://mlsanity.com/paper/2207.04236/) (2022)
- [Polarization Wavefront Lidar: Learning Large Scene Reconstruction from Polarized Wavefronts](https://mlsanity.com/paper/2406.03461/) (2024)
- [Robust Depth Enhancement via Polarization Prompt Fusion Tuning](https://mlsanity.com/paper/2404.04318/) (2024)
- [MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets](https://mlsanity.com/paper/2404.13923/) (2024)
- [Deep Polarization Cues for Single-shot Shape and Subsurface Scattering Estimation](https://mlsanity.com/paper/2407.08149/) (2024)
- [Learning to Deblur Polarized Images](https://mlsanity.com/paper/2402.18134/) (2024)
- [Polarimetric Spatio-Temporal Light Transport Probing](https://mlsanity.com/paper/2105.11609/) (2021)
- [Differentiable Display Photometric Stereo](https://mlsanity.com/paper/2306.13325/) (2023)
- [Uncertainty for SVBRDF Acquisition using Frequency Analysis](https://mlsanity.com/paper/2406.17774/) (2024)
- [A Real-world Display Inverse Rendering Dataset](https://mlsanity.com/paper/2508.14411/) (2025)
- [SS-SfP:Neural Inverse Rendering for Self Supervised Shape from (Mixed) Polarization](https://mlsanity.com/paper/2407.09294/) (2024)
- [Polarimetric BSSRDF Acquisition of Dynamic Faces](https://mlsanity.com/paper/2501.01980/) (2024)

## Abstract (arXiv)

We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints (known shape or multiview acquisition), we lift such restrictions by coupling polarization imaging with deep learning to achieve high quality estimate of 3D object shape (surface normals and depth) and SVBRDF using single-view polarization imaging under frontal flash illumination. In addition to acquired polarization images, we provide our deep network with strong novel cues related to shape and reflectance, in the form of a normalized Stokes map and an estimate of diffuse color. We additionally describe modifications to network architecture and training loss which provide further qualitative improvements. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.