# Neural 3D Reconstruction in the Wild

arXiv: https://arxiv.org/abs/2205.12955 · HTML: https://mlsanity.com/paper/2205.12955/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 64 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 30 of 38)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Urban Radiance Fields](https://mlsanity.com/paper/2111.14643/) (2021)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields](https://mlsanity.com/paper/2106.13228/) (2021)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [Neural Rerendering in the Wild](https://mlsanity.com/paper/1904.04290/) (2019)

## Built on by (most cited first, 30 of 79)

- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [High-quality Surface Reconstruction using Gaussian Surfels](https://mlsanity.com/paper/2404.17774/) (2024)
- [ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields](https://mlsanity.com/paper/2211.11704/) (2022)
- [NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance](https://mlsanity.com/paper/2312.00846/) (2023)
- [GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction](https://mlsanity.com/paper/2309.02436/) (2023)
- [StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views](https://mlsanity.com/paper/2306.04988/) (2023)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields](https://mlsanity.com/paper/2402.00864/) (2024)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis](https://mlsanity.com/paper/2210.00647/) (2022)
- [Vox-Surf: Voxel-based Implicit Surface Representation](https://mlsanity.com/paper/2208.10925/) (2022)
- [SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations](https://mlsanity.com/paper/2210.02299/) (2022)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction](https://mlsanity.com/paper/2405.19671/) (2024)
- [Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting](https://mlsanity.com/paper/2406.11672/) (2024)
- [NeRF-HuGS: Improved Neural Radiance Fields in Non-static Scenes Using Heuristics-Guided Segmentation](https://mlsanity.com/paper/2403.17537/) (2024)
- [VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors](https://mlsanity.com/paper/2407.02945/) (2024)
- [Dynamic LiDAR Re-simulation using Compositional Neural Fields](https://mlsanity.com/paper/2312.05247/) (2023)
- [DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery](https://mlsanity.com/paper/2503.16964/) (2025)
- [Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting](https://mlsanity.com/paper/2408.09130/) (2024)
- [Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflections](https://mlsanity.com/paper/2304.08706/) (2023)
- [SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes](https://mlsanity.com/paper/2310.13030/) (2023)
- [NeRFs in Robotics: A Survey](https://mlsanity.com/paper/2405.01333/) (2024)
- [PuzzleAvatar: Assembling 3D Avatars from Personal Albums](https://mlsanity.com/paper/2405.14869/) (2024)
- [PersonNeRF: Personalized Reconstruction from Photo Collections](https://mlsanity.com/paper/2302.08504/) (2023)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container](https://mlsanity.com/paper/2303.13805/) (2023)

## Abstract (arXiv)

We are witnessing an explosion of neural implicit representations in computer vision and graphics. Their applicability has recently expanded beyond tasks such as shape generation and image-based rendering to the fundamental problem of image-based 3D reconstruction. However, existing methods typically assume constrained 3D environments with constant illumination captured by a small set of roughly uniformly distributed cameras. We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections in the presence of varying illumination. To achieve this, we propose a hybrid voxel- and surface-guided sampling technique that allows for more efficient ray sampling around surfaces and leads to significant improvements in reconstruction quality. Further, we present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes. We perform extensive experiments, demonstrating that our approach surpasses both classical and neural reconstruction methods on a wide variety of metrics.