# Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflections

arXiv: https://arxiv.org/abs/2304.08706 · HTML: https://mlsanity.com/paper/2304.08706/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 67 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 30 of 32)

- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction](https://mlsanity.com/paper/2205.15848/) (2022)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [Neural Volumes: Learning Dynamic Renderable Volumes from Images](https://mlsanity.com/paper/1906.07751/) (2019)
- [Improving neural implicit surfaces geometry with patch warping](https://mlsanity.com/paper/2112.09648/) (2021)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [NeRFReN: Neural Radiance Fields with Reflections](https://mlsanity.com/paper/2111.15234/) (2021)
- [HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details](https://mlsanity.com/paper/2206.07850/) (2022)
- [Single Image Reflection Separation with Perceptual Losses](https://mlsanity.com/paper/1806.05376/) (2018)
- [Neural 3D Reconstruction in the Wild](https://mlsanity.com/paper/2205.12955/) (2022)
- [Multi-View Mesh Reconstruction with Neural Deferred Shading](https://mlsanity.com/paper/2212.04386/) (2022)
- [Location-aware Single Image Reflection Removal](https://mlsanity.com/paper/2012.07131/) (2020)
- [Robust Reflection Removal with Reflection-free Flash-only Cues](https://mlsanity.com/paper/2103.04273/) (2021)
- [Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes](https://mlsanity.com/paper/2004.10904/) (2020)
- [S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint](https://mlsanity.com/paper/2210.08936/) (2022)
- [Differentiable Stereopsis: Meshes from multiple views using differentiable rendering](https://mlsanity.com/paper/2110.05472/) (2021)
- [Multi-view 3D Reconstruction of a Texture-less Smooth Surface of Unknown Generic Reflectance](https://mlsanity.com/paper/2105.11599/) (2021)
- [Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction](https://mlsanity.com/paper/2203.12613/) (2022)
- [A Categorized Reflection Removal Dataset with Diverse Real-world Scenes](https://mlsanity.com/paper/2108.03380/) (2021)

## Built on by (most cited first, 19 of 19)

- [StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views](https://mlsanity.com/paper/2306.04988/) (2023)
- [Revisiting Single Image Reflection Removal In the Wild](https://mlsanity.com/paper/2311.17320/) (2023)
- [DC-Gaussian: Improving 3D Gaussian Splatting for Reflective Dash Cam Videos](https://mlsanity.com/paper/2405.17705/) (2024)
- [LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow](https://mlsanity.com/paper/2409.05688/) (2024)
- [Diffusion-Based Depth Inpainting for Transparent and Reflective Objects](https://mlsanity.com/paper/2410.08567/) (2024)
- [Seeing and Seeing Through the Glass: Real and Synthetic Data for Multi-Layer Depth Estimation](https://mlsanity.com/paper/2503.11633/) (2025)
- [MS-NeRF: Multi-Space Neural Radiance Fields](https://mlsanity.com/paper/2305.04268/) (2023)
- [PG-NeuS: Robust and Efficient Point Guidance for Multi-View Neural Surface Reconstruction](https://mlsanity.com/paper/2310.07997/) (2023)
- [Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats](https://mlsanity.com/paper/2410.02764/) (2024)
- [Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression](https://mlsanity.com/paper/2307.06143/) (2023)
- [REF2-NeRF: Reflection and Refraction aware Neural Radiance Field](https://mlsanity.com/paper/2311.17116/) (2023)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [From Transparent to Opaque: Rethinking Neural Implicit Surfaces with α-NeuS](https://mlsanity.com/paper/2411.05362/) (2024)
- [RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objects](https://mlsanity.com/paper/2505.05848/) (2025)
- [TR-Gaussians: High-fidelity Real-time Rendering of Planar Transmission and Reflection with 3D Gaussian Splatting](https://mlsanity.com/paper/2511.13009/) (2025)
- [TransmissiveGS: Residual-Guided Disentangled Gaussian Splatting for Transmissive Scene Reconstruction and Rendering](https://mlsanity.com/paper/2605.10705/) (2026)
- [SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Grouping](https://mlsanity.com/paper/2605.28735/) (2026)
- [Principled Reflection Separation via Nonlinear Superposition and Feature Interaction](https://mlsanity.com/paper/2606.02831/) (2026)
- [Reflection Separation from a Single Image via Joint Latent Diffusion](https://mlsanity.com/paper/2606.04107/) (2026)

## Abstract (arXiv)

Neural implicit methods have achieved high-quality 3D object surfaces under slight specular highlights. However, high specular reflections (HSR) often appear in front of target objects when we capture them through glasses. The complex ambiguity in these scenes violates the multi-view consistency, then makes it challenging for recent methods to reconstruct target objects correctly. To remedy this issue, we present a novel surface reconstruction framework, NeuS-HSR, based on implicit neural rendering. In NeuS-HSR, the object surface is parameterized as an implicit signed distance function (SDF). To reduce the interference of HSR, we propose decomposing the rendered image into two appearances: the target object and the auxiliary plane. We design a novel auxiliary plane module by combining physical assumptions and neural networks to generate the auxiliary plane appearance. Extensive experiments on synthetic and real-world datasets demonstrate that NeuS-HSR outperforms state-of-the-art approaches for accurate and robust target surface reconstruction against HSR. Code is available at https://github.com/JiaxiongQ/NeuS-HSR.