# Consistent Video Depth Estimation

arXiv: https://arxiv.org/abs/2004.15021 · HTML: https://mlsanity.com/paper/2004.15021/

## Key facts

- Cited by 220 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 30 of 220)

- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024)
- [Shape of Motion: 4D Reconstruction from a Single Video](https://mlsanity.com/paper/2407.13764/) (2024)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos](https://mlsanity.com/paper/2412.04463/) (2024)
- [DynIBaR: Neural Dynamic Image-Based Rendering](https://mlsanity.com/paper/2211.11082/) (2022)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos](https://mlsanity.com/paper/2409.02095/) (2024)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time](https://mlsanity.com/paper/2202.08614/) (2022)
- [SceneScape: Text-Driven Consistent Scene Generation](https://mlsanity.com/paper/2302.01133/) (2023)
- [NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields](https://mlsanity.com/paper/2210.15947/) (2022)
- [Editable Free-viewpoint Video Using a Layered Neural Representation](https://mlsanity.com/paper/2104.14786/) (2021)
- [DynamicStereo: Consistent Dynamic Depth from Stereo Videos](https://mlsanity.com/paper/2305.02296/) (2023)
- [DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing](https://mlsanity.com/paper/2403.17822/) (2024)
- [Layered Neural Atlases for Consistent Video Editing](https://mlsanity.com/paper/2109.11418/) (2021)
- [TTT3R: 3D Reconstruction as Test-Time Training](https://mlsanity.com/paper/2509.26645/) (2025)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields](https://mlsanity.com/paper/2305.11588/) (2023)
- [Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset](https://mlsanity.com/paper/2107.04286/) (2021)
- [Towards An End-to-End Framework for Flow-Guided Video Inpainting](https://mlsanity.com/paper/2204.02663/) (2022)
- [Progressively Optimized Local Radiance Fields for Robust View Synthesis](https://mlsanity.com/paper/2303.13791/) (2023)

## Abstract (arXiv)

We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video. We leverage a conventional structure-from-motion reconstruction to establish geometric constraints on pixels in the video. Unlike the ad-hoc priors in classical reconstruction, we use a learning-based prior, i.e., a convolutional neural network trained for single-image depth estimation. At test time, we fine-tune this network to satisfy the geometric constraints of a particular input video, while retaining its ability to synthesize plausible depth details in parts of the video that are less constrained. We show through quantitative validation that our method achieves higher accuracy and a higher degree of geometric consistency than previous monocular reconstruction methods. Visually, our results appear more stable. Our algorithm is able to handle challenging hand-held captured input videos with a moderate degree of dynamic motion. The improved quality of the reconstruction enables several applications, such as scene reconstruction and advanced video-based visual effects.