# NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images

arXiv: https://arxiv.org/abs/2406.07111 · HTML: https://mlsanity.com/paper/2406.07111/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization](https://mlsanity.com/paper/2303.07418/) (2023)
- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2206.05737/) (2022)
- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details](https://mlsanity.com/paper/2206.07850/) (2022)
- [Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection](https://mlsanity.com/paper/2303.10840/) (2023)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [Polarimetric Multi-View Inverse Rendering](https://mlsanity.com/paper/2007.08830/) (2020)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces](https://mlsanity.com/paper/2303.17712/) (2023)
- [Multi-View Azimuth Stereo via Tangent Space Consistency](https://mlsanity.com/paper/2303.16447/) (2023)

## Built on by (most cited first, 22 of 22)

- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [PolGS: Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstruction](https://mlsanity.com/paper/2509.19726/) (2025)
- [PolarAnything: Diffusion-based Polarimetric Image Synthesis](https://mlsanity.com/paper/2507.17268/) (2025)
- [Benchmarking Burst Super-Resolution for Polarization Images: Noise Dataset and Analysis](https://mlsanity.com/paper/2503.18705/) (2025)
- [MultimodalStudio: A Heterogeneous Sensor Dataset and Framework for Neural Rendering across Multiple Imaging Modalities](https://mlsanity.com/paper/2503.19673/) (2025)
- [Glossy Object Reconstruction with Cost-effective Polarized Acquisition](https://mlsanity.com/paper/2504.07025/) (2025)
- [Reflect3r: Single-View 3D Stereo Reconstruction Aided by Mirror Reflections](https://mlsanity.com/paper/2509.20607/) (2025)
- [Thermal Polarimetric Multi-view Stereo](https://mlsanity.com/paper/2510.20972/) (2025)
- [PMNI: Pose-free Multi-view Normal Integration for Reflective and Textureless Surface Reconstruction](https://mlsanity.com/paper/2504.08410/) (2025)
- [RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objects](https://mlsanity.com/paper/2505.05848/) (2025)
- [PolarGS: Polarimetric Cues for Ambiguity-Free Gaussian Splatting with Accurate Geometry Recovery](https://mlsanity.com/paper/2512.00794/) (2025)
- [Architectural Unification for Polarimetric Imaging Across Multiple Degradations](https://mlsanity.com/paper/2603.05834/) (2026)
- [PolarAPP: Beyond Polarization Demosaicking for Polarimetric Applications](https://mlsanity.com/paper/2603.23071/) (2026)
- [PolarVLM: Bridging the Semantic-Physical Gap in Vision-Language Models](https://mlsanity.com/paper/2605.07574/) (2026)
- [PolarGuide-GSDR: 3D Gaussian Splatting Driven by Polarization Priors and Deferred Reflection for Real-World Reflective Scenes](https://mlsanity.com/paper/2512.02664/) (2025)
- [EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups](https://mlsanity.com/paper/2602.24065/) (2026)
- [Revisiting Shape from Polarization in the Era of Vision Foundation Models](https://mlsanity.com/paper/2603.04817/) (2026)
- [PolGS++: Physically-Guided Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstruction](https://mlsanity.com/paper/2603.10801/) (2026)
- [PhyGaP: Physically-Grounded Gaussians with Polarization Cues](https://mlsanity.com/paper/2603.14001/) (2026)
- [Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimation](https://mlsanity.com/paper/2603.27891/) (2026)
- [CDPR: Cross-modal Diffusion with Polarization for Reliable Monocular Depth Estimation](https://mlsanity.com/paper/2604.11097/) (2026)
- [Differentiable Polarized Path Tracing](https://mlsanity.com/paper/2607.13265/) (2026)

## Abstract (arXiv)

We present NeRSP, a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. Reflective surface reconstruction is extremely challenging as specular reflections are view-dependent and thus violate the multiview consistency for multiview stereo. On the other hand, sparse image inputs, as a practical capture setting, commonly cause incomplete or distorted results due to the lack of correspondence matching. This paper jointly handles the challenges from sparse inputs and reflective surfaces by leveraging polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency, which jointly optimize the surface geometry modeled via implicit neural representation. Based on the experiments on our synthetic and real datasets, we achieve the state-of-the-art surface reconstruction results with only 6 views as input.