Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
Not on the map. Why?
Abstract (arXiv)
We present a fully data-driven method to compute depth from diverse monocular video sequences that contain large amounts of non-rigid objects, e.g., people. In order to learn reconstruction cues for non-rigid scenes, we introduce a new dataset consisting of stereo videos scraped in-the-wild. This dataset has a wide variety of scene types, and features large amounts of nonrigid objects, especially people. From this, we compute disparity maps to be used as supervision to train our approach. We propose a loss function that allows us to generate a depth prediction even with unknown camera intrinsics and stereo baselines in the dataset. We validate the use of large amounts of Internet video by evaluating our method on existing video datasets with depth supervision, including SINTEL, and KITTI, and show that our approach generalizes better to natural scenes.
Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 14; 2022: 14; 2023: 20; 2024: 14; 2025: 6; 2026: 5.
73 papers.
Sort:
By area:
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2022Kubric: A scalable dataset generatorcited by 373
- 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
- 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 173
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
- 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 147
- 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
- 2021Neural Trajectory Fields for Dynamic Novel View Synthesiscited by 85
- 2021Multi-Task Self-Training for Learning General Representationscited by 85
- 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 73
38 more
- 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Taskscited by 72
- 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
- 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
- 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
- 2021Consistent Depth of Moving Objects in Videocited by 56
- 2022Hierarchical Normalization for Robust Monocular Depth Estimationcited by 51
- 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 50
- 2021Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Predictioncited by 48
- 2023RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptionscited by 47
- 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
- 2023DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Modelcited by 43
- 2021Toward Practical Monocular Indoor Depth Estimationcited by 42
- 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Imagecited by 42
- 2023Flow supervision for Deformable NeRFcited by 42
- 2023EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimationcited by 42
- 2021Self-Supervised Multi-Frame Monocular Scene Flowcited by 29
- 2021Improving 360 Monocular Depth Estimation via Non-local Dense Prediction Transformer and Joint Supervised and Self-supervised Learningcited by 19
- 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 18
- 2022Improving Monocular Visual Odometry Using Learned Depthcited by 17
- 2022Depth Map Decomposition for Monocular Depth Estimationcited by 16
- 2024ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimationcited by 15
- 2022PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentationcited by 14
- 2023PODIA-3D: Domain Adaptation of 3D Generative Model Across Large Domain Gap Using Pose-Preserved Text-to-Image Diffusioncited by 12
- 2023The Second Monocular Depth Estimation Challengecited by 12
- 2023Temporally Consistent Online Depth Estimation Using Point-Based Fusioncited by 12
- 2023SAAM: Stealthy Adversarial Attack on Monocular Depth Estimationcited by 12
- 2022A Survey on RGB-D Datasetscited by 11
- 2022Towards 3D Scene Reconstruction from Locally Scale-Aligned Monocular Video Depthcited by 11
- 2021Embedding Novel Views in a Single JPEG Imagecited by 10
- 2024The Third Monocular Depth Estimation Challengecited by 10
- 2021Estimating Image Depth in the Comics Domaincited by 8
- 2021THE Benchmark: Transferable Representation Learning for Monocular Height Estimationcited by 8
- 2023Salient Object Detection in RGB-D Videoscited by 8
- 2024ImmersePro: End-to-End Stereo Video Synthesis Via Implicit Disparity Learningcited by 8
- 2025The Fourth Monocular Depth Estimation Challengecited by 8
- 2023Single-Stage 3D Geometry-Preserving Depth Estimation Model Training on Dataset Mixtures with Uncalibrated Stereo Datacited by 6
- 2023NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimationcited by 6
- 2023Robust Geometry-Preserving Depth Estimation Using Differentiable Renderingcited by 6
50 most cited of 73 All 73 (JSON)
- 2026Disparity Has a Sign: Stereo Matching Beyond the Zero-Disparity Plane6 Sep 2026
- 2026Monocular Depth Estimation from a Single Image: Progress and Opportunities1 Sep 2026
- 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device9 Jul 2026
- 2026SciFlow: Semantic Cross Interference for Self-Supervised Optical Flow Domain Generalization27 Jun 2026
- 2026Pixel-Perfect Visual Geometry Estimation8 Jan 2026
- 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers8 Oct 2025
- 2025EfficientDepth: A Fast and Detail-Preserving Monocular Depth Estimation Model26 Sep 2025
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation25 Jun 2025
- 2025The Fourth Monocular Depth Estimation Challenge24 Apr 2025
- 2025One Look is Enough: Seamless Patchwise Refinement for Zero-Shot Monocular Depth Estimation on High-Resolution Images28 Mar 2025
- 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimation5 Jan 2025
- 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation18 Dec 2024
38 more
- 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos12 Dec 2024
- 2024STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimation2 Dec 2024
- 2024DissolveStereo: Coarse Depth Injection for Zero-Shot Stereo Video Generation21 Nov 2024
- 2024DELTA: Dense Efficient Long-range 3D Tracking for any video31 Oct 2024
- 2024ImmersePro: End-to-End Stereo Video Synthesis Via Implicit Disparity Learning30 Sep 2024
- 2024Self-Distilled Depth Refinement with Noisy Poisson Fusion26 Sep 2024
- 2024Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective6 Sep 2024
- 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos3 Sep 2024
- 2024Floating No More: Object-Ground Reconstruction from a Single Image26 Jul 2024
- 2024ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation11 Jul 2024
- 2024The Third Monocular Depth Estimation Challenge25 Apr 2024
- 2024The Devil is in the Edges: Monocular Depth Estimation with Edge-aware Consistency Fusion30 Mar 2024
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data19 Jan 2024
- 2023GenDepth: Generalizing Monocular Depth Estimation for Arbitrary Camera Parameters via Ground Plane Embedding10 Dec 2023
- 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks30 Oct 2023
- 2023Salient Object Detection in RGB-D Videos24 Oct 2023
- 2023RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions23 Oct 2023
- 2023Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering18 Sep 2023
- 2023SAAM: Stealthy Adversarial Attack on Monocular Depth Estimation6 Aug 2023
- 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation26 Jul 2023
- 2023NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation17 Jul 2023
- 2023Background Prompting for Improved Object Depth8 Jun 2023
- 2023Single-Stage 3D Geometry-Preserving Depth Estimation Model Training on Dataset Mixtures with Uncalibrated Stereo Data5 Jun 2023
- 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videos3 May 2023
- 2023EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimation16 Apr 2023
- 2023Temporally Consistent Online Depth Estimation Using Point-Based Fusion15 Apr 2023
- 2023The Second Monocular Depth Estimation Challenge14 Apr 2023
- 2023Impact of Pseudo Depth on Open World Object Segmentation with Minimal User Guidance12 Apr 2023
- 2023DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Model6 Apr 2023
- 2023PODIA-3D: Domain Adaptation of 3D Generative Model Across Large Domain Gap Using Pose-Preserved Text-to-Image Diffusion4 Apr 2023
- 2023Flow supervision for Deformable NeRF28 Mar 2023
- 2023Versatile Depth Estimator Based on Common Relative Depth Estimation and Camera-Specific Relative-to-Metric Depth Conversion20 Mar 2023
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth23 Feb 2023
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov 2022
- 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes7 Nov 2022
- 2022Hierarchical Normalization for Robust Monocular Depth Estimation18 Oct 2022
- 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image28 Aug 2022
- 2022Depth Map Decomposition for Monocular Depth Estimation23 Aug 2022
50 newest of 73 All 73 (JSON)
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
- 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
- 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 18
- 2023PODIA-3D: Domain Adaptation of 3D Generative Model Across Large Domain Gap Using Pose-Preserved Text-to-Image Diffusioncited by 12
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimationcited by 6
- 2024DissolveStereo: Coarse Depth Injection for Zero-Shot Stereo Video Generationcited by 5
- 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers8 Oct 2025
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation25 Jun 2025
- 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimation5 Jan 2025
- 2024DissolveStereo: Coarse Depth Injection for Zero-Shot Stereo Video Generation21 Nov 2024
- 2023PODIA-3D: Domain Adaptation of 3D Generative Model Across Large Domain Gap Using Pose-Preserved Text-to-Image Diffusion4 Apr 2023
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov 2022
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2022Kubric: A scalable dataset generatorcited by 373
- 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
- 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 173
- 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 147
- 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
- 2021Neural Trajectory Fields for Dynamic Novel View Synthesiscited by 85
- 2021Multi-Task Self-Training for Learning General Representationscited by 85
- 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 73
- 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Taskscited by 72
12 most cited of 67 All 67 (JSON)
- 2026Disparity Has a Sign: Stereo Matching Beyond the Zero-Disparity Plane6 Sep 2026
- 2026Monocular Depth Estimation from a Single Image: Progress and Opportunities1 Sep 2026
- 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device9 Jul 2026
- 2026SciFlow: Semantic Cross Interference for Self-Supervised Optical Flow Domain Generalization27 Jun 2026
- 2026Pixel-Perfect Visual Geometry Estimation8 Jan 2026
- 2025EfficientDepth: A Fast and Detail-Preserving Monocular Depth Estimation Model26 Sep 2025
- 2025The Fourth Monocular Depth Estimation Challenge24 Apr 2025
- 2025One Look is Enough: Seamless Patchwise Refinement for Zero-Shot Monocular Depth Estimation on High-Resolution Images28 Mar 2025
- 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation18 Dec 2024
- 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos12 Dec 2024
- 2024STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimation2 Dec 2024
- 2024DELTA: Dense Efficient Long-range 3D Tracking for any video31 Oct 2024
12 newest of 67 All 67 (JSON)