# Revisiting Shape from Polarization in the Era of Vision Foundation Models

arXiv: https://arxiv.org/abs/2603.04817 · HTML: https://mlsanity.com/paper/2603.04817/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 44 of its 78 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 30 of 44)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision](https://mlsanity.com/paper/2410.19115/) (2024)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details](https://mlsanity.com/paper/2507.02546/) (2025)
- [StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal](https://mlsanity.com/paper/2406.16864/) (2024)
- [Rethinking Inductive Biases for Surface Normal Estimation](https://mlsanity.com/paper/2403.00712/) (2024)
- [Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think](https://mlsanity.com/paper/2409.11355/) (2024)
- [Polarized Reflection Removal with Perfect Alignment in the Wild](https://mlsanity.com/paper/2003.12789/) (2020)
- [Designing Deep Networks for Surface Normal Estimation](https://mlsanity.com/paper/1411.4958/) (2014)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentation](https://mlsanity.com/paper/2408.01343/) (2024)
- [Marr Revisited: 2D-3D Alignment via Surface Normal Prediction](https://mlsanity.com/paper/1604.01347/) (2016)
- [Surface Normal Estimation of Tilted Images via Spatial Rectifier](https://mlsanity.com/paper/2007.09264/) (2020)
- [Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset](https://mlsanity.com/paper/2504.08541/) (2025)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images](https://mlsanity.com/paper/2406.07111/) (2024)
- [PolarFree: Polarization-based Reflection-free Imaging](https://mlsanity.com/paper/2503.18055/) (2025)
- [Transparent Shape from a Single View Polarization Image](https://mlsanity.com/paper/2204.06331/) (2022)
- [PISR: Polarimetric Neural Implicit Surface Reconstruction for Textureless and Specular Objects](https://mlsanity.com/paper/2409.14331/) (2024)
- [All-photon Polarimetric Time-of-Flight Imaging](https://mlsanity.com/paper/2112.09278/) (2021)

## Abstract (arXiv)

We show that, with polarization cues, a lightweight model trained on a small dataset can outperform RGB-only vision foundation models (VFMs) in single-shot object-level surface normal estimation. Shape from polarization (SfP) has long been studied due to the strong physical relationship between polarization and surface geometry. Meanwhile, driven by scaling laws, RGB-only VFMs trained on large datasets have recently achieved impressive performance and surpassed existing SfP methods. This situation raises questions about the necessity of polarization cues, which require specialized hardware and have limited training data. We argue that the weaker performance of prior SfP methods does not come from the polarization modality itself, but from domain gaps. These domain gaps mainly arise from two sources. First, existing synthetic datasets use limited and unrealistic 3D objects, with simple geometry and random texture maps that do not match the underlying shapes. Second, real-world polarization signals are often affected by sensor noise, which is not well modeled during training. To address the first issue, we render a high-quality polarization dataset using 1,954 3D-scanned real-world objects. We further incorporate pretrained DINOv3 priors to improve generalization to unseen objects. To address the second issue, we introduce polarization sensor-aware data augmentation that better reflects real-world conditions. With only 40K training scenes, our method significantly outperforms both state-of-the-art SfP approaches and RGB-only VFMs. Extensive experiments show that polarization cues enable a 33x reduction in training data or an 8x reduction in model parameters, while still achieving better performance than RGB-only counterparts.