# Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimation

arXiv: https://arxiv.org/abs/2603.27891 · HTML: https://mlsanity.com/paper/2603.27891/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 29 Mar 2026.

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details](https://mlsanity.com/paper/2507.02546/) (2025)
- [Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction](https://mlsanity.com/paper/2409.18124/) (2024)
- [High-quality Surface Reconstruction using Gaussian Surfels](https://mlsanity.com/paper/2404.17774/) (2024)
- [StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal](https://mlsanity.com/paper/2406.16864/) (2024)
- [Rethinking Inductive Biases for Surface Normal Estimation](https://mlsanity.com/paper/2403.00712/) (2024)
- [Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think](https://mlsanity.com/paper/2409.11355/) (2024)
- [What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?](https://mlsanity.com/paper/2403.06090/) (2024)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstruction](https://mlsanity.com/paper/2406.05774/) (2024)
- [Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion](https://mlsanity.com/paper/2412.13389/) (2024)
- [GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation](https://mlsanity.com/paper/2012.06980/) (2020)
- [Surface Normal Estimation of Tilted Images via Spatial Rectifier](https://mlsanity.com/paper/2007.09264/) (2020)
- [Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation](https://mlsanity.com/paper/2311.18363/) (2023)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images](https://mlsanity.com/paper/2406.07111/) (2024)
- [Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model](https://mlsanity.com/paper/2512.01030/) (2025)
- [PISR: Polarimetric Neural Implicit Surface Reconstruction for Textureless and Specular Objects](https://mlsanity.com/paper/2409.14331/) (2024)
- [Fast constrained sampling in pre-trained diffusion models](https://mlsanity.com/paper/2410.18804/) (2024)
- [Linear Differential Constraints for Photo-polarimetric Height Estimation](https://mlsanity.com/paper/1708.07718/) (2017)
- [Robust Depth Enhancement via Polarization Prompt Fusion Tuning](https://mlsanity.com/paper/2404.04318/) (2024)

## Abstract (arXiv)

Monocular surface normal estimators trained on large-scale RGB-normal data often perform poorly in the edge cases of reflective, textureless, and dark surfaces. Polarization encodes surface orientation independently of texture and albedo, offering a physics-based complement for these cases. Existing polarization methods, however, require multi-view capture or specialized training data, limiting generalization. We introduce Poppy, a training-free framework that refines normals from any frozen RGB backbone using single-shot polarization measurements at test time. Keeping backbone weights frozen, Poppy optimizes per-pixel offsets to the input RGB and output normal along with a learned reflectance decomposition. A differentiable rendering layer converts the refined normals into polarization predictions and penalizes mismatches with the observed signal. Across seven benchmarks and three backbone architectures (diffusion, flow, and feed-forward), Poppy reduces mean angular error by 23-26% on synthetic data and 6-16% on real data. These results show that guiding learned RGB-based normal estimators with polarization cues at test time refines normals on challenging surfaces without retraining.