mlsanityData to 14 Oct 2026

Less is More: Consistent Video Depth Estimation with Masked Frames Modeling

arXiv 2208.00380 ↗First posted 31 Jul 2022cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

Temporal consistency is the key challenge of video depth estimation. Previous works are based on additional optical flow or camera poses, which is time-consuming. By contrast, we derive consistency with less information. Since videos inherently exist with heavy temporal redundancy, a missing frame could be recovered from neighboring ones. Inspired by this, we propose the frame masking network (FMNet), a spatial-temporal transformer network predicting the depth of masked frames based on their neighboring frames. By reconstructing masked temporal features, the FMNet can learn intrinsic inter-frame correlations, which leads to consistency. Compared with prior arts, experimental results demonstrate that our approach achieves comparable spatial accuracy and higher temporal consistency without any additional information. Our work provides a new perspective on consistent video depth estimation. Our official project page is https://github.com/RaymondWang987/FMNet.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 47 of its 66 references that are in the map. First posted on arXiv 31 Jul 2022.

Builds on

47 of 66 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2022Generative Adversarial Networkscited by 5,627
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
35 more
  1. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  4. 2021Vision Transformers for Dense Predictioncited by 1,803
  5. 2021ViViT: A Video Vision Transformercited by 1,620
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  7. 2015Spatial Transformer Networkscited by 1,429
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  9. 2021Multiscale Vision Transformerscited by 897
  10. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  11. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  12. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  13. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 465
  14. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  15. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  16. 2019Structured Knowledge Distillation for Dense Predictioncited by 279
  17. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 234
  18. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  19. 2020Consistent Video Depth Estimationcited by 220
  20. 2020Robust Consistent Video Depth Estimationcited by 188
  21. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  22. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  23. 2018BA-Net: Dense Bundle Adjustment Networkcited by 175
  24. 2018Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Networkcited by 110
  25. 2016Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networkscited by 91
  26. 2020Don't Forget The Past: Recurrent Depth Estimation from Monocular Videocited by 71
  27. 2019Exploiting temporal consistency for real-time video depth estimationcited by 61
  28. 2019DepthTransfer: Depth Extraction from Video Using Non-parametric Samplingcited by 60
  29. 2021Consistent Depth of Moving Objects in Videocited by 56
  30. 2022BokehMe: When Neural Rendering Meets Classical Renderingcited by 48
  31. 2021Deep Two-View Structure-from-Motion Revisitedcited by 39
  32. 2018Deep attention-based classification network for robust depth predictioncited by 36
  33. 2020ConvTransformer: A Convolutional Transformer Network for Video Frame Synthesiscited by 34
  34. 2019Synthetic Defocus and Look-Ahead Autofocus for Casual Videographycited by 28
  35. 2018Unsupervised Learning of Monocular Depth Estimation with Bundle Adjustment, Super-Resolution and Clip Losscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 9; 2025: 8; 2026: 5.

30 papers.

Sort:

By area:

  1. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 207
  2. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
  3. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  4. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  5. 2024Scaling 4D Representationscited by 30
  6. 2024Video Depth without Video Modelscited by 29
  7. 20233D Cinemagraphy from a Single Imagecited by 24
  8. 2023MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimationcited by 24
  9. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 14
  10. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  11. 2024FutureDepth: Learning to Predict the Future Improves Video Depth Estimationcited by 13
  12. 2023Temporally Consistent Online Depth Estimation Using Point-Based Fusioncited by 12
18 more
  1. 2024Match Stereo Videos via Bidirectional Alignmentcited by 11
  2. 2024Buffer Anytime: Zero-Shot Video Depth and Normal from Image Priorscited by 11
  3. 2025TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusioncited by 9
  4. 2026From Frames to Sequences: Temporally Consistent Human-Centric Dense Predictioncited by 9
  5. 2023Constraining Depth Map Geometry for Multi-View Stereo: A Dual-Depth Approach with Saddle-shaped Depth Cellscited by 8
  6. 2023Diffusion-Augmented Depth Prediction with Sparse Annotationscited by 8
  7. 2023NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimationcited by 6
  8. 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimationcited by 6
  9. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  10. 2025SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusioncited by 4
  11. 2023Aerial Image Object Detection With Vision Transformer Detector (ViTDet)cited by 3
  12. 2025Self-Guided Masked Autoencodercited by 3
  13. 2025Video Depth Propagationcited by 3
  14. 2026Depth Completion as Parameter-Efficient Test-Time Adaptationcited by 3
  15. 2025GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusioncited by 2
  16. 2024STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimationcited by 1
  17. 2026Spectral-Geometric Neural Fields for Pose-Free LiDAR View Synthesiscited by 0
  18. 2026GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depthcited by 0