# Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection

arXiv: https://arxiv.org/abs/2303.10840 · HTML: https://mlsanity.com/paper/2303.10840/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction](https://mlsanity.com/paper/2205.15848/) (2022)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [Improving neural implicit surfaces geometry with patch warping](https://mlsanity.com/paper/2112.09648/) (2021)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [ActiveNeRF: Learning where to See with Uncertainty Estimation](https://mlsanity.com/paper/2209.08546/) (2022)
- [Multi-Scale Geometric Consistency Guided Multi-View Stereo](https://mlsanity.com/paper/1904.08103/) (2019)
- [Learning Signed Distance Field for Multi-view Surface Reconstruction](https://mlsanity.com/paper/2108.09964/) (2021)
- [Seeing the World in a Bag of Chips](https://mlsanity.com/paper/2001.04642/) (2020)
- [Multi-view 3D Reconstruction of a Texture-less Smooth Surface of Unknown Generic Reflectance](https://mlsanity.com/paper/2105.11599/) (2021)
- [nLMVS-Net: Deep Non-Lambertian Multi-View Stereo](https://mlsanity.com/paper/2207.11876/) (2022)

## Built on by (most cited first, 30 of 69)

- [LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching](https://mlsanity.com/paper/2311.11284/) (2023)
- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading](https://mlsanity.com/paper/2404.09412/) (2024)
- [Neural Microfacet Fields for Inverse Rendering](https://mlsanity.com/paper/2303.17806/) (2023)
- [ReTR: Modeling Rendering Via Transformer for Generalizable Neural Surface Reconstruction](https://mlsanity.com/paper/2305.18832/) (2023)
- [SpecNeRF: Gaussian Directional Encoding for Specular Reflections](https://mlsanity.com/paper/2312.13102/) (2023)
- [UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections](https://mlsanity.com/paper/2312.13285/) (2023)
- [SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes](https://mlsanity.com/paper/2410.17249/) (2024)
- [MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference](https://mlsanity.com/paper/2510.11387/) (2025)
- [NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images](https://mlsanity.com/paper/2406.07111/) (2024)
- [GeoSplatting: Towards Geometry Guided Gaussian Splatting for Physically-based Inverse Rendering](https://mlsanity.com/paper/2410.24204/) (2024)
- [EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting](https://mlsanity.com/paper/2405.14959/) (2024)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields](https://mlsanity.com/paper/2403.16224/) (2024)
- [TransparentGS: Fast Inverse Rendering of Transparent Objects with Gaussians](https://mlsanity.com/paper/2504.18768/) (2025)
- [Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects](https://mlsanity.com/paper/2305.17929/) (2023)
- [PISR: Polarimetric Neural Implicit Surface Reconstruction for Textureless and Specular Objects](https://mlsanity.com/paper/2409.14331/) (2024)
- [Geometry Field Splatting with Gaussian Surfels](https://mlsanity.com/paper/2411.17067/) (2024)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [X-Ray: A Sequential 3D Representation For Generation](https://mlsanity.com/paper/2404.14329/) (2024)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [SpectralGaussians: Semantic, spectral 3D Gaussian splatting for multi-spectral scene representation, visualization and analysis](https://mlsanity.com/paper/2408.06975/) (2024)
- [BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2408.13370/) (2024)
- [FlexGen: Flexible Multi-View Generation from Text and Image Inputs](https://mlsanity.com/paper/2410.10745/) (2024)
- [TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through Neural Radiance Fields](https://mlsanity.com/paper/2310.10650/) (2023)
- [PFGS: High Fidelity Point Cloud Rendering via Feature Splatting](https://mlsanity.com/paper/2407.03857/) (2024)
- [AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction](https://mlsanity.com/paper/2410.01202/) (2024)
- [GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors](https://mlsanity.com/paper/2403.11899/) (2024)
- [Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling](https://mlsanity.com/paper/2406.16360/) (2024)
- [PRM: Photometric Stereo based Large Reconstruction Model](https://mlsanity.com/paper/2412.07371/) (2024)

## Abstract (arXiv)

Neural implicit surface learning has shown significant progress in multi-view 3D reconstruction, where an object is represented by multilayer perceptrons that provide continuous implicit surface representation and view-dependent radiance. However, current methods often fail to accurately reconstruct reflective surfaces, leading to severe ambiguity. To overcome this issue, we propose Ref-NeuS, which aims to reduce ambiguity by attenuating the effect of reflective surfaces. Specifically, we utilize an anomaly detector to estimate an explicit reflection score with the guidance of multi-view context to localize reflective surfaces. Afterward, we design a reflection-aware photometric loss that adaptively reduces ambiguity by modeling rendered color as a Gaussian distribution, with the reflection score representing the variance. We show that together with a reflection direction-dependent radiance, our model achieves high-quality surface reconstruction on reflective surfaces and outperforms the state-of-the-arts by a large margin. Besides, our model is also comparable on general surfaces.