mlsanityData to 7 Oct 2026

Towards 3D Scene Reconstruction from Locally Scale-Aligned Monocular Video Depth

arXiv 2202.01470 ↗First posted 3 Feb 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Existing monocular depth estimation methods have achieved excellent robustness in diverse scenes, but they can only retrieve affine-invariant depth, up to an unknown scale and shift. However, in some video-based scenarios such as video depth estimation and 3D scene reconstruction from a video, the unknown scale and shift residing in per-frame prediction may cause the depth inconsistency. To solve this problem, we propose a locally weighted linear regression method to recover the scale and shift with very sparse anchor points, which ensures the scale consistency along consecutive frames. Extensive experiments show that our method can boost the performance of existing state-of-the-art approaches by 50% at most over several zero-shot benchmarks. Besides, we merge over 6.3 million RGBD images to train strong and robust depth models. Our produced ResNet50-backbone model even outperforms the state-of-the-art DPT ViT-Large model. Combining with geometry-based reconstruction methods, we formulate a new dense 3D scene reconstruction pipeline, which benefits from both the scale consistency of sparse points and the robustness of monocular methods. By performing the simple per-frame prediction over a video, the accurate 3D scene shape can be recovered.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 57 of its 83 references that are in the map. First posted on arXiv 3 Feb 2022.

Builds on

57 of 83 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2021Vision Transformers for Dense Predictioncited by 1,789
  9. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  10. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  11. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
38 more
  1. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  2. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  3. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  4. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  5. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  6. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  7. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  8. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  9. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  10. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
  11. 2021Neural RGB-D Surface Reconstructioncited by 312
  12. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  13. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  14. 2020Consistent Video Depth Estimationcited by 220
  15. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  16. 2020Non-Local Spatial Propagation Network for Depth Completioncited by 194
  17. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  18. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  19. 2020Robust Consistent Video Depth Estimationcited by 188
  20. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  21. 2016Single-Image Depth Perception in the Wildcited by 179
  22. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  23. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  24. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  25. 2018Learning Depth with Convolutional Spatial Propagation Networkcited by 135
  26. 2019DPSNet: End-to-end Deep Plane Sweep Stereocited by 115
  27. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  28. 2019CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completioncited by 110
  29. 2019Web Stereo Video Supervision for Depth Prediction from Dynamic Scenescited by 73
  30. 2019Neural RGB->D Sensing: Depth and Uncertainty from a Video Cameracited by 69
  31. 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
  32. 2020DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusioncited by 68
  33. 2020OASIS: A Large-Scale Dataset for Single Image 3D in the Wildcited by 62
  34. 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
  35. 2020DiverseDepth: Affine-invariant Depth Prediction Using Diverse Datacited by 60
  36. 2020Multi-view Depth Estimation using Epipolar Spatio-Temporal Networkscited by 59
  37. 2019Dense Depth Posterior (DDP) from Single Image and Sparse Rangecited by 57
  38. 2021Consistent Depth of Moving Objects in Videocited by 56

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 3; 2026: 3.

11 papers.

  1. 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Imagecited by 42
  2. 2025Jasmine: Harnessing Diffusion Prior for Self-supervised Depth Estimationcited by 20
  3. 2023FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Modelscited by 17
  4. 2024DiffCalib: Reformulating Monocular Camera Calibration as Diffusion-Based Dense Incident Map Generationcited by 9
  5. 2022Towards Domain-agnostic Depth Completioncited by 6
  6. 2026Large Depth Completion Model from Sparse Observationscited by 3
  7. 2025Monocular One-Shot Metric-Depth Alignment for RGB-Based Robot Graspingcited by 2
  8. 2023Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraintscited by 1
  9. 2026Learning Image-Adaptive Scale Fields for Metric Depth Recoverycited by 1
  10. 2025Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Camerascited by 0
  11. 2026Test-Time Adaptation for Height Completion via Self-Supervised ViT Features and Monocular Foundation Modelscited by 0