mlsanityData to 7 Oct 2026

DoubleTake: Geometry Guided Depth Estimation

arXiv 2406.18387 ↗First posted 26 Jun 2024cs.CV cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Estimating depth from a sequence of posed RGB images is a fundamental computer vision task, with applications in augmented reality, path planning etc. Prior work typically makes use of previous frames in a multi view stereo framework, relying on matching textures in a local neighborhood. In contrast, our model leverages historical predictions by giving the latest 3D geometry data as an extra input to our network. This self-generated geometric hint can encode information from areas of the scene not covered by the keyframes and it is more regularized when compared to individual predicted depth maps for previous frames. We introduce a Hint MLP which combines cost volume features with a hint of the prior geometry, rendered as a depth map from the current camera location, together with a measure of the confidence in the prior geometry. We demonstrate that our method, which can run at interactive speeds, achieves state-of-the-art estimates of depth and 3D scene reconstruction in both offline and incremental evaluation scenarios.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 63 of its 82 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

63 of 82 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
  5. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  6. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  7. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  8. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  9. 2020Convolutional Occupancy Networkscited by 760
  10. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  11. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
  12. 2018Pyramid Stereo Matching Networkcited by 486
38 more
  1. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  2. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 464
  3. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  4. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  5. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  6. 2017Sparsity Invariant CNNscited by 325
  7. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  8. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  9. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 242
  10. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  11. 2020Consistent Video Depth Estimationcited by 220
  12. 2022Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstructioncited by 211
  13. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  14. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
  15. 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 159
  16. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  17. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  18. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 144
  19. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  20. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  21. 2018Learning Depth with Convolutional Spatial Propagation Networkcited by 135
  22. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  23. 2019DPSNet: End-to-end Deep Plane Sweep Stereocited by 115
  24. 2019Domain-invariant Stereo Matching Networkscited by 108
  25. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 107
  26. 2022SimpleRecon: 3D Reconstruction Without 3D Convolutionscited by 83
  27. 2020DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusioncited by 68
  28. 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
  29. 2018MVDepthNet: Real-time Multiview Depth Estimation Neural Networkcited by 59
  30. 2020Multi-view Depth Estimation using Epipolar Spatio-Temporal Networkscited by 59
  31. 2023Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedracited by 57
  32. 2023NeRFMeshing: Distilling Neural Radiance Fields into Geometrically-Accurate 3D Meshescited by 52
  33. 2022Multiview Stereo with Cascaded Epipolar RAFTcited by 51
  34. 2019Multi-View Stereo by Temporal Nonparametric Fusioncited by 46
  35. 2021Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completioncited by 39
  36. 2021VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusioncited by 38
  37. 2023SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimatescited by 35
  38. 2022Sparsity Agnostic Depth Completioncited by 27

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 3.

9 papers.

  1. 2025MVSAnywhere: Zero-Shot Multi-View Stereocited by 26
  2. 2024Learning-based Multi-View Stereo: A Surveycited by 18
  3. 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimationcited by 6
  4. 2025Synthetic Human Action Video Data Generation with Pose Transfercited by 2
  5. 2026LiDAR Prompted Spatio-Temporal Multi-View Stereo for Autonomous Drivingcited by 2
  6. 2025An Embodied AR Navigation Agent: Integrating BIM with Retrieval-Augmented Generation for Language Guidancecited by 0
  7. 2025UniFField: A Generalizable Unified Neural Feature Field for Visual, Semantic, and Spatial Uncertainties in Any Scenecited by 0
  8. 2026VFM-Recon: Unlocking Cross-Domain Scene-Level Neural Reconstruction with Scale-Aligned Foundation Priorscited by 0
  9. 2026LiteMVS: Efficient Multi-View Stereo with Foundation Distillation and Expert Aggregationcited by 0