mlsanityData to 7 Oct 2026

Align3R: Aligned Monocular Depth Estimation for Dynamic Videos

arXiv 2412.03079 ↗First posted 4 Dec 2024cs.CVcited by 79

Not on the map. Why?

Abstract (arXiv)

Recent developments in monocular depth estimation methods enable high-quality depth estimation of single-view images but fail to estimate consistent video depth across different frames. Recent works address this problem by applying a video diffusion model to generate video depth conditioned on the input video, which is training-expensive and can only produce scale-invariant depth values without camera poses. In this paper, we propose a novel video-depth estimation method called Align3R to estimate temporal consistent depth maps for a dynamic video. Our key idea is to utilize the recent DUSt3R model to align estimated monocular depth maps of different timesteps. First, we fine-tune the DUSt3R model with additional estimated monocular depth as inputs for the dynamic scenes. Then, we apply optimization to reconstruct both depth maps and camera poses. Extensive experiments demonstrate that Align3R estimates consistent video depth and camera poses for a monocular video with superior performance than baseline methods.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 56 of its 68 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

56 of 68 references.

References · 56

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  7. 2024Depth Anything V2cited by 1,385
  8. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  9. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  10. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  11. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  12. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
38 more
  1. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  2. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  3. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  4. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  5. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  6. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  7. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  8. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  9. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  10. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  11. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  12. 20243D Reconstruction with Spatial Memorycited by 269
  13. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  14. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  15. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  16. 2020Consistent Video Depth Estimationcited by 220
  17. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  18. 2020Robust Consistent Video Depth Estimationcited by 188
  19. 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
  20. 2022Deep Patch Visual Odometrycited by 186
  21. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  22. 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 180
  23. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  24. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  25. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  26. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  27. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
  28. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 151
  29. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  30. 2021Transformer-Based Attention Networks for Continuous Pixel-Wise Predictioncited by 125
  31. 2022P3Depth: Monocular Depth Estimation with a Piecewise Planarity Priorcited by 106
  32. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  33. 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 90
  34. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Thinkcited by 82
  35. 2024Depth Any Video with Scalable Synthetic Datacited by 57
  36. 2021Consistent Depth of Moving Objects in Videocited by 56
  37. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  38. 2024MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priorscited by 51

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 46; 2026: 33.

79 papers.

Sort:

By area:

  1. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  2. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  3. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  4. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  5. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  6. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 57
  7. 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 46
  8. 2025LONG3R: Long Sequence Streaming 3D Reconstructioncited by 45
  9. 2026LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstructioncited by 32
  10. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 31
  11. 2026V-DPM: 4D Video Reconstruction with Dynamic Point Mapscited by 29
  12. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
38 more
  1. 2026VGGT-Omegacited by 20
  2. 2025Test3R: Learning to Reconstruct 3D at Test Timecited by 19
  3. 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backendcited by 16
  4. 2025LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Mergingcited by 16
  5. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 15
  6. 2025FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolutioncited by 14
  7. 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videoscited by 13
  8. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  9. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  10. 2025D2USt3R: Enhancing 3D Reconstruction for Dynamic Scenescited by 10
  11. 2025Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentationcited by 10
  12. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 10
  13. 2026PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagerycited by 10
  14. 2025UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimationcited by 9
  15. 2025EgoM2P: Egocentric Multimodal Multitask Pretrainingcited by 9
  16. 2025Co-Me: Confidence-Guided Token Merging for Visual Geometric Transformerscited by 9
  17. 2025Review of Feed-forward 3D Reconstruction: From DUSt3R to VGGTcited by 8
  18. 2025VGGT-X: When VGGT Meets Dense Novel View Synthesiscited by 8
  19. 2026MoRe: Motion-aware Feed-forward 4D Reconstruction Transformercited by 8
  20. 2025PanoSplatt3R: Leveraging Perspective Pretraining for Generalized Unposed Wide-Baseline Panorama Reconstructioncited by 7
  21. 2025PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perceptioncited by 7
  22. 2026UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Passcited by 7
  23. 2025E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Modelscited by 6
  24. 2025LHM++: An Efficient Large Human Reconstruction Model for Pose-free Images to 3Dcited by 6
  25. 2025TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixelscited by 6
  26. 2025Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillationcited by 5
  27. 2025LASER: Layer-wise Scale Alignment for Training-Free Streaming 4D Reconstructioncited by 5
  28. 2025MVInverse: Feed-forward Multi-view Inverse Rendering in Secondscited by 5
  29. 2025CoCo4D: Comprehensive and Complex 4D Scene Generationcited by 4
  30. 2026CoMoVi: Co-Generation of 3D Human Motions and Realistic Videoscited by 4
  31. 2026UFO-4D: Unposed Feedforward 4D Reconstruction from Two Imagescited by 4
  32. 2025Beyond the Final Layer: Hierarchical Query Fusion Transformer with Agent-Interpolation Initialization for 3D Instance Segmentationcited by 3
  33. 2025Mem4D: Decoupling Static and Dynamic Memory for Dynamic Scene Reconstructioncited by 3
  34. 2026MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAEcited by 3
  35. 2026FreeArtGS: Articulated Gaussian Splatting Under Free-moving Scenariocited by 3
  36. 2026Self-Improving 4D Perception via Self-Distillationcited by 3
  37. 2026Towards Consistent Video Geometry Estimationcited by 3
  38. 2025UAV4D: Dynamic Neural Rendering of Human-Centric UAV Imagery using Gaussian Splattingcited by 2

50 most cited of 79 All 79 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation
  2. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
  3. 2026ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning
  4. 2024DepthART: Monocular Depth Estimation as Autoregressive Refinement Task
  5. 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis
  6. 2025DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation
  7. 2026DVD: Deterministic Video Depth Estimation with Generative Priors
  8. 2024Video Depth without Video Models