mlsanityData to 7 Oct 2026

Neural RGB-D Surface Reconstruction

arXiv 2104.04532 ↗First posted 9 Apr 2021cs.CVcited by 312

Not on the map. Why?

Abstract (arXiv)

Obtaining high-quality 3D reconstructions of room-scale scenes is of paramount importance for upcoming applications in AR or VR. These range from mixed reality applications for teleconferencing, virtual measuring, virtual room planing, to robotic applications. While current volume-based view synthesis methods that use neural radiance fields (NeRFs) show promising results in reproducing the appearance of an object or scene, they do not reconstruct an actual surface. The volumetric representation of the surface based on densities leads to artifacts when a surface is extracted using Marching Cubes, since during optimization, densities are accumulated along the ray and are not used at a single sample point in isolation. Instead of this volumetric representation of the surface, we propose to represent the surface using an implicit function (truncated signed distance function). We show how to incorporate this representation in the NeRF framework, and extend it to use depth measurements from a commodity RGB-D sensor, such as a Kinect. In addition, we propose a pose and camera refinement technique which improves the overall reconstruction quality. In contrast to concurrent work on integrating depth priors in NeRF which concentrates on novel view synthesis, our approach is able to reconstruct high-quality, metrical 3D reconstructions.

Key facts. Cited by 312 papers in the map (data through 7 Oct 2026). Builds on 52 of its 96 references that are in the map. First posted on arXiv 9 Apr 2021.

Builds on

52 of 96 references.

References · 52

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  12. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
38 more
  1. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  2. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  3. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  4. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  5. 2020Convolutional Occupancy Networkscited by 760
  6. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  7. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  10. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  11. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  12. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  13. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  14. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  15. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  16. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  17. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  18. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 381
  19. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  20. 2020State of the Art on Neural Renderingcited by 276
  21. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  22. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  23. 2021DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networkscited by 232
  24. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  25. 2017Learning a Multi-View Stereo Machinecited by 194
  26. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  27. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  28. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  29. 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 151
  30. 2020DeepFactors: Real-Time Probabilistic Dense Monocular SLAMcited by 117
  31. 2016BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integrationcited by 116
  32. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  33. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  34. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  35. 2020RoutedFusion: Learning Real-time Depth Map Fusioncited by 52
  36. 2020NeuralFusion: Online Depth Fusion in Latent Spacecited by 46
  37. 2019SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representationscited by 37
  38. 2020SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scanscited by 34

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 38; 2023: 76; 2024: 59; 2025: 60; 2026: 69.

312 papers.

Sort:

By area:

  1. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  2. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  3. 2021Neural Fields in Visual Computing and Beyondcited by 539
  4. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  5. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  6. 2021Advances in Neural Renderingcited by 365
  7. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  8. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
  9. 20243D Reconstruction with Spatial Memorycited by 269
  10. 2021Urban Radiance Fieldscited by 263
  11. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  12. 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
38 more
  1. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  2. 2023Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAMcited by 183
  3. 2023Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Imagescited by 172
  4. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  5. 2023Point-SLAM: Dense Neural Point Cloud-based SLAMcited by 160
  6. 2022ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fieldscited by 158
  7. 2022Instant Volumetric Head Avatarscited by 157
  8. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  9. 2023Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splattingcited by 152
  10. 2022Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representationcited by 149
  11. 2025Streaming 4D Visual Geometry Transformercited by 147
  12. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  13. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  14. 2023GO-SLAM: Global Optimization for Consistent 3D Instant Reconstructioncited by 132
  15. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  16. 2022iSDF: Real-Time Neural Signed Distance Fields for Robot Perceptioncited by 107
  17. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 82
  18. 2021DiGS : Divergence guided shape implicit neural representation for unoriented point cloudscited by 76
  19. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  20. 2024How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Surveycited by 72
  21. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  22. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  23. 2024Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussianscited by 64
  24. 2023PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustmentcited by 62
  25. 2022GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstructioncited by 61
  26. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
  27. 2022Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Cameracited by 57
  28. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layoutcited by 57
  29. 2026InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streamscited by 51
  30. 2022Vox-Surf: Voxel-based Implicit Surface Representationcited by 49
  31. 2024Loopy-SLAM: Dense Neural SLAM with Loop Closurescited by 48
  32. 2022CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimizationcited by 46
  33. 2022Low-Rank Tensor Function Representation for Multi-Dimensional Data Recoverycited by 45
  34. 2022RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Modelscited by 45
  35. 2025LONG3R: Long Sequence Streaming 3D Reconstructioncited by 45
  36. 2022SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representationscited by 44
  37. 2023Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motioncited by 44
  38. 2022A Level Set Theory for Neural Implicit Evolution under Explicit Flowscited by 42

50 most cited of 312 All 312 (JSON)