mlsanityData to 7 Oct 2026

Learning Temporally Consistent Video Depth from Video Diffusion Priors

arXiv 2406.01493 ↗First posted 3 Jun 2024cs.CVcited by 100

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

This work addresses the challenge of streamed video depth estimation, which expects not only per-frame accuracy but, more importantly, cross-frame consistency. We argue that sharing contextual information between frames or clips is pivotal in fostering temporal consistency. Therefore, we reformulate depth prediction into a conditional generation problem to provide contextual information within a clip and across clips. Specifically, we propose a consistent context-aware training and inference strategy for arbitrarily long videos to provide cross-clip context. We sample independent noise levels for each frame within a clip during training while using a sliding window strategy and initializing overlapping frames with previously predicted frames without adding noise. Moreover, we design an effective training strategy to provide context within a clip. Extensive experimental results validate our design choices and demonstrate the superiority of our approach, dubbed ChronoDepth. Project page: https://xdimlab.github.io/ChronoDepth/.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 75 of its 93 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

75 of 93 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matching20 Mar
  2. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar
  3. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation4 Dec
  4. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation2 Jun
  5. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 70

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
38 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2021Vision Transformers for Dense Predictioncited by 1,789
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  7. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  10. 2021Variational Diffusion Modelscited by 1,145
  11. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  12. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  13. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  14. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  15. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  16. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  17. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  18. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  19. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  20. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  21. 2023Zero-shot Image-to-Image Translationcited by 503
  22. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  23. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  24. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  25. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  26. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  27. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  28. 2020Virtual KITTI 2cited by 345
  29. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  30. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  31. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  32. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  33. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  34. 2020Consistent Video Depth Estimationcited by 220
  35. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  36. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  37. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  38. 2020Robust Consistent Video Depth Estimationcited by 188

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 58; 2026: 28.

100 papers.

Sort:

By area:

  1. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  2. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  3. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  4. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 202
  5. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  6. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  7. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 88
  8. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  9. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  10. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  11. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  12. 2024Depth Any Video with Scalable Synthetic Datacited by 57
38 more
  1. 2024Gramian Multimodal Representation Learning and Alignmentcited by 52
  2. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  3. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  4. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  5. 2024Generative Omnimatte: Learning to Decompose Video into Layerscited by 38
  6. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  7. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  8. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priorscited by 36
  9. 2025AI-Generated Video Detection via Perceptual Straighteningcited by 34
  10. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  11. 2024Video Depth without Video Modelscited by 28
  12. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  13. 2025MVSAnywhere: Zero-Shot Multi-View Stereocited by 26
  14. 2025NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priorscited by 25
  15. 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
  16. 2025POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstructioncited by 24
  17. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  18. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  19. 2025Multi-view Reconstruction via SfM-guided Monocular Depth Estimationcited by 17
  20. 20264RC: 4D Reconstruction via Conditional Querying Anytime and Anywherecited by 17
  21. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  22. 2025FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolutioncited by 14
  23. 2024X-Drive: Cross-modality consistent multi-sensor data synthesis for driving scenarioscited by 13
  24. 2025Seurat: From Moving Points to Depthcited by 13
  25. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  26. 2025Depth AnyEvent: A Cross-Modal Distillation Paradigm for Event-Based Monocular Depth Estimationcited by 13
  27. 2025Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusioncited by 12
  28. 2025L4P: Towards Unified Low-Level 4D Vision Perceptioncited by 12
  29. 2024Buffer Anytime: Zero-Shot Video Depth and Normal from Image Priorscited by 11
  30. 2025Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpaintingcited by 11
  31. 2025DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generationcited by 11
  32. 2025TACO: Taming Diffusion for in-the-wild Video Amodal Completioncited by 10
  33. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  34. 2025M2SVid: End-to-End Inpainting and Refinement for Monocular-to-Stereo Video Conversioncited by 10
  35. 2025UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimationcited by 9
  36. 2025Tracktention: Leveraging Point Tracking to Attend Videos Faster and Bettercited by 8
  37. 2025The Fourth Monocular Depth Estimation Challengecited by 8
  38. 2025One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Controlcited by 8

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Video Diffusion Models with Local-Global Context Guidance
  2. 2024DissolveStereo: Coarse Depth Injection for Zero-Shot Stereo Video Generation
  3. 2026DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation
  4. 2025Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimation
  5. 2024Training-Free Semantic Video Composition via Pre-trained Diffusion Model
  6. 2026PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference
  7. 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions
  8. 2026Temporal Aware Pruning for Efficient Diffusion-based Video Generation