mlsanityData to 7 Oct 2026

NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video

arXiv 2104.00681 ↗First posted 1 Apr 2021cs.CV cs.ROcited by 241

Not on the map. Why?

Abstract (arXiv)

We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video. Unlike previous methods that estimate single-view depth maps separately on each key-frame and fuse them later, we propose to directly reconstruct local surfaces represented as sparse TSDF volumes for each video fragment sequentially by a neural network. A learning-based TSDF fusion module based on gated recurrent units is used to guide the network to fuse features from previous fragments. This design allows the network to capture local smoothness prior and global shape prior of 3D surfaces when sequentially reconstructing the surfaces, resulting in accurate, coherent, and real-time surface reconstruction. The experiments on ScanNet and 7-Scenes datasets show that our system outperforms state-of-the-art methods in terms of both accuracy and speed. To the best of our knowledge, this is the first learning-based system that is able to reconstruct dense coherent 3D geometry in real-time.

Key facts. Cited by 241 papers in the map (data through 7 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

35 of 59 references.

References · 35

  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  7. 2020Neural Sparse Voxel Fieldscited by 959
  8. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  9. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  10. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  11. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  12. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
23 more
  1. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  2. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  3. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  4. 2020Consistent Video Depth Estimationcited by 220
  5. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  6. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  7. 2017Learning a Multi-View Stereo Machinecited by 194
  8. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  9. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  10. 2019Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awarenesscited by 151
  11. 2017SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsiscited by 128
  12. 2019DPSNet: End-to-end Deep Plane Sweep Stereocited by 115
  13. 2020Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation and Gauss-Newton Refinementcited by 105
  14. 2018DeepTAM: Deep Tracking and Mappingcited by 87
  15. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  16. 2019Neural RGB->D Sensing: Depth and Uncertainty from a Video Cameracited by 69
  17. 2018MVDepthNet: Real-time Multiview Depth Estimation Neural Networkcited by 59
  18. 2020RoutedFusion: Learning Real-time Depth Map Fusioncited by 52
  19. 2019Multi-View Stereo by Temporal Nonparametric Fusioncited by 46
  20. 2020NeuralFusion: Online Depth Fusion in Latent Spacecited by 46
  21. 2019A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensorscited by 44
  22. 2020Occlusion-Aware Depth Estimation with Adaptive Normal Constraintscited by 41
  23. 2020SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsiscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 37; 2023: 65; 2024: 66; 2025: 33; 2026: 30.

241 papers.

Sort:

By area:

  1. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  2. 2021Neural Fields in Visual Computing and Beyondcited by 539
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  4. 2021Neural RGB-D Surface Reconstructioncited by 312
  5. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  6. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  7. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  8. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 198
  9. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  10. 2022SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Viewscited by 175
  11. 2023Point-SLAM: Dense Neural Point Cloud-based SLAMcited by 160
  12. 2022ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fieldscited by 158
38 more
  1. 2023Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splattingcited by 152
  2. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  3. 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 141
  4. 2022NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priorscited by 125
  5. 2022Neural 3D Scene Reconstruction with the Manhattan-world Assumptioncited by 123
  6. 2024Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splatscited by 111
  7. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 107
  8. 2022NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstructioncited by 99
  9. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  10. 2024FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenescited by 86
  11. 2022SimpleRecon: 3D Reconstruction Without 3D Convolutionscited by 83
  12. 2022Neural 3D Reconstruction in the Wildcited by 79
  13. 2022DP-NeRF: Deblurred Neural Radiance Field with Physical Scene Priorscited by 70
  14. 2023Neural Map Prior for Autonomous Drivingcited by 68
  15. 2024Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussianscited by 64
  16. 2022NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologiescited by 62
  17. 2023PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustmentcited by 62
  18. 2022GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstructioncited by 61
  19. 2022VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstructioncited by 61
  20. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  21. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  22. 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 56
  23. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
  24. 2023NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detectioncited by 51
  25. 2024Loopy-SLAM: Dense Neural SLAM with Loop Closurescited by 48
  26. 2022RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Modelscited by 45
  27. 2021VolumeFusion: Deep Depth Fusion for 3D Scene Reconstructioncited by 44
  28. 2022SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representationscited by 44
  29. 2023Dense RGB SLAM with Neural Implicit Mapscited by 42
  30. 2023I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFscited by 42
  31. 2024GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstructioncited by 39
  32. 2021VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusioncited by 38
  33. 2022Fast Learning Radiance Fields by Shooting Much Fewer Rayscited by 38
  34. 2023SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understandingcited by 38
  35. 2024GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAMcited by 36
  36. 2021Input-level Inductive Biases for 3D Reconstructioncited by 31
  37. 2023SurfelNeRF: Neural Surfel Radiance Fields for Online Photorealistic Reconstruction of Indoor Scenescited by 30
  38. 2023SGAligner : 3D Scene Alignment with Scene Graphscited by 30

50 most cited of 241 All 241 (JSON)