mlsanityData to 7 Oct 2026

DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos

arXiv 2409.02095 ↗First posted 3 Sep 2024cs.CV cs.AI cs.GRcited by 202

Not on the map. Why?

Cited by 58 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Estimating video depth in open-world scenarios is challenging due to the diversity of videos in appearance, content motion, camera movement, and length. We present DepthCrafter, an innovative method for generating temporally consistent long depth sequences with intricate details for open-world videos, without requiring any supplementary information such as camera poses or optical flow. The generalization ability to open-world videos is achieved by training the video-to-depth model from a pre-trained image-to-video diffusion model, through our meticulously designed three-stage training strategy. Our training approach enables the model to generate depth sequences with variable lengths at one time, up to 110 frames, and harvest both precise depth details and rich content diversity from realistic and synthetic datasets. We also propose an inference strategy that can process extremely long videos through segment-wise estimation and seamless stitching. Comprehensive evaluations on multiple datasets reveal that DepthCrafter achieves state-of-the-art performance in open-world video depth estimation under zero-shot settings. Furthermore, DepthCrafter facilitates various downstream applications, including depth-based visual effects and conditional video generation.

Key facts. Cited by 202 papers in the map (data through 7 Oct 2026). Builds on 65 of its 79 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

65 of 79 references.

References · 65

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  12. 2022Video Diffusion Modelscited by 2,280
38 more
  1. 2021Vision Transformers for Dense Predictioncited by 1,789
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  3. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2024Depth Anything V2cited by 1,385
  6. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  8. 2023Vision Transformers Need Registerscited by 844
  9. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  10. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  11. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  12. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  13. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  14. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  15. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  16. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  17. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  18. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  19. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  20. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  21. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  22. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  23. 2020Consistent Video Depth Estimationcited by 220
  24. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  25. 2020Robust Consistent Video Depth Estimationcited by 188
  26. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  27. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  28. 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 180
  29. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  30. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  31. 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 147
  32. 2023Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fieldscited by 142
  33. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  34. 2021Transformer-Based Attention Networks for Continuous Pixel-Wise Predictioncited by 125
  35. 2022DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimationcited by 116
  36. 2022P3Depth: Monocular Depth Estimation with a Piecewise Planarity Priorcited by 106
  37. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  38. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 103; 2026: 71.

202 papers.

Sort:

By area:

  1. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  2. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  3. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  4. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 202
  5. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  6. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  7. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  8. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  9. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  10. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 88
  11. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  12. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
38 more
  1. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  2. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  3. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  4. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  5. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  6. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  7. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 61
  8. 2024Depth Any Video with Scalable Synthetic Datacited by 57
  9. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  10. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  11. 2025GenFusion: Closing the Loop between Reconstruction and Generation via Videoscited by 41
  12. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 39
  13. 2024Generative Omnimatte: Learning to Decompose Video into Layerscited by 38
  14. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  15. 2024StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videoscited by 36
  16. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priorscited by 36
  17. 2025GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priorscited by 36
  18. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  19. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 33
  20. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 33
  21. 2025EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Meshcited by 32
  22. 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 32
  23. 2024Video Depth without Video Modelscited by 28
  24. 2024DepthLab: From Partial to Completecited by 27
  25. 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Videocited by 27
  26. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
  27. 2025MVSAnywhere: Zero-Shot Multi-View Stereocited by 26
  28. 2025NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priorscited by 25
  29. 2025POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstructioncited by 24
  30. 2025Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformerscited by 24
  31. 2024Novel View Extrapolation with Video Diffusion Priorscited by 23
  32. 2025Back on Track: Bundle Adjustment for Dynamic Scene Reconstructioncited by 23
  33. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  34. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  35. 2025OmniVDiff: Omni Controllable Video Diffusion for Generation and Understandingcited by 22
  36. 2025UniFuture: A 4D Driving World Model for Future Generation and Perceptioncited by 21
  37. 2024Towards Precise Scaling Laws for Video Diffusion Transformerscited by 20
  38. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 20

50 most cited of 202 All 202 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception
  2. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observations
  3. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control
  4. 2026SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models
  5. 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
  6. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matching
  7. 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions
  8. 2024Owl-1: Omni World Model for Consistent Long Video Generation