DeepV2D: Video to Depth with Differentiable Structure from Motion
Not on the map. Why?
Abstract (arXiv)
We propose DeepV2D, an end-to-end deep learning architecture for predicting depth from video. DeepV2D combines the representation ability of neural networks with the geometric principles governing image formation. We compose a collection of classical geometric algorithms, which are converted into trainable modules and combined into an end-to-end differentiable architecture. DeepV2D interleaves two stages: motion estimation and depth estimation. During inference, motion and depth estimation are alternated and converge to accurate depth. Code is available https://github.com/princeton-vl/DeepV2D.
Key facts. Cited by 185 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 20; 2022: 29; 2023: 29; 2024: 42; 2025: 43; 2026: 22.
185 papers.
Sort:
By area:
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 286
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
- 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
- 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
38 more
- 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
- 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 192
- 2023NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAMcited by 164
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
- 2024DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshingcited by 138
- 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 136
- 2023GO-SLAM: Global Optimization for Consistent 3D Instant Reconstructioncited by 133
- 2022NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priorscited by 125
- 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 115
- 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
- 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
- 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
- 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
- 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
- 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 72
- 2024Deep Patch Visual SLAMcited by 71
- 2021Monocular 3D Object Detection: An Extrinsic Parameter Free Approachcited by 69
- 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 65
- 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 65
- 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
- 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
- 2024Depth Any Video with Scalable Synthetic Datacited by 57
- 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 56
- 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
- 2021Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometrycited by 46
- 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
- 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 44
- 2023Dense RGB SLAM with Neural Implicit Mapscited by 42
- 2023Flow supervision for Deformable NeRFcited by 42
- 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
- 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 40
- 2021Deep Two-View Structure-from-Motion Revisitedcited by 39
- 2022A Benchmark and a Baseline for Robust Multi-view Depth Estimationcited by 39
- 2023R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Camerascited by 39
- 2021SAIL-VOS 3D: A Synthetic Dataset and Baselines for Object Detection and 3D Mesh Reconstruction from Video Datacited by 38
- 2021MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 38
- 2022RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimizationcited by 38
- 2022Monocular 3D Object Detection with Depth from Motioncited by 38
50 most cited of 185 All 185 (JSON)
- 2026CLoSeR: Closing the Loop for Long-Context Streaming Reconstruction1 Oct 2026
- 2026OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Compute28 Sep 2026
- 2026Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering18 Sep 2026
- 2026VGGT-GS SLAM: Uncalibrated Monocular Gaussian Splatting SLAM with Feed-Forward Priors17 Sep 2026
- 2026DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers11 Sep 2026
- 2026UniSim-SLAM: Feed-Forward SLAM with Unified Sim(3) Optimization3 Aug 2026
- 2026Glob3R: Global Structure-from-Motion with 3D Foundation Models10 Jul 2026
- 2026Planar-SfM: Camera Pose Estimation via Homography Graph Embeddings30 Jun 2026
- 2026MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry15 Jun 2026
- 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application2 Jun 2026
- 2026Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence19 May 2026
- 2026VGGT-Omega14 May 2026
38 more
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM24 Apr 2026
- 2026Online3R: Online Learning for Consistent Sequential Reconstruction Based on Geometry Foundation Model10 Apr 2026
- 2026VGGT-SLAM++8 Apr 2026
- 2026FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction19 Mar 2026
- 2026UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perception26 Feb 2026
- 2026Learning Proposes, Geometry Disposes: A Modular Framework for Efficient Spatial Reasoning16 Feb 2026
- 2026S-MUSt3R: Sliding Multi-view 3D Reconstruction4 Feb 2026
- 2026Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardware31 Jan 2026
- 2026VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstruction27 Jan 2026
- 2025FoundationSLAM: Unleashing the Power of Depth Foundation Models for End-to-End Dense Visual SLAM31 Dec 2025
- 2025Emergent Outlier View Rejection in Visual Geometry Grounded Transformers3 Dec 2025
- 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend25 Nov 2025
- 2025Depth Anything 3: Recovering the Visual Space from Any Views13 Nov 2025
- 2025Lumos3D: A Single-Forward Framework for Low-Light 3D Scene Restoration12 Nov 2025
- 2025Policies over Poses: Reinforcement Learning based Distributed Pose-Graph Optimization for Multi-Robot SLAM26 Oct 2025
- 2025MoRe: Monocular Geometry Refinement via Graph Optimization for Cross-View Consistency8 Oct 2025
- 2025Dropping the D: RGB-D SLAM Without the Depth Sensor7 Oct 2025
- 2025Monocular Visual Odometry without Calibration or Test-time Optimization2 Oct 2025
- 2025Stylos: Multi-View 3D Stylization with Single-Forward Gaussian Splatting30 Sep 2025
- 2025From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representations28 Sep 2025
- 2025SLAM-Former: Putting SLAM into One Transformer21 Sep 2025
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstruction16 Sep 2025
- 2025SAIL-Recon: Large SfM by Augmenting Scene Regression with Localization25 Aug 2025
- 2025HAMSt3R: Human-Aware Multi-view Stereo 3D Reconstruction22 Aug 2025
- 2025Reconstructing 4D Spatial Intelligence: A Survey28 Jul 2025
- 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences22 Jul 2025
- 2025DINO-VO: A Feature-based Visual Odometry Leveraging a Visual Foundation Model17 Jul 2025
- 2025DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation2 Jul 2025
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation25 Jun 2025
- 2025Multimodal Fusion SLAM with Fourier Attention22 Jun 2025
- 2025Test3R: Learning to Reconstruct 3D at Test Time16 Jun 2025
- 2025UNO: Unified Self-Supervised Monocular Odometry for Platform-Agnostic Deployment8 Jun 2025
- 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planning7 Jun 2025
- 2025On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images5 Jun 2025
- 2025UAV4D: Dynamic Neural Rendering of Human-Centric UAV Imagery using Gaussian Splatting5 Jun 2025
- 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold18 May 2025
- 2025Dynamic Camera Poses and Where to Find Them24 Apr 2025
50 newest of 185 All 185 (JSON)
- 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
- 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 65
- 2024Depth Any Video with Scalable Synthetic Datacited by 57
- 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimationcited by 6
- 2024FiffDepth: Feed-forward Transformation of Diffusion-Based Generators for Detailed Depth Estimationcited by 5
- 2025DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimationcited by 2
- 2025DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation2 Jul 2025
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation25 Jun 2025
- 2025Can Video Diffusion Model Reconstruct 4D Geometry?27 Mar 2025
- 2024World-consistent Video Diffusion with Explicit 3D Modeling2 Dec 2024
- 2024FiffDepth: Feed-forward Transformation of Diffusion-Based Generators for Detailed Depth Estimation1 Dec 2024
- 2024Depth Any Video with Scalable Synthetic Data14 Oct 2024
- 2024Learning Temporally Consistent Video Depth from Video Diffusion Priors3 Jun 2024
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 286
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
- 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
- 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
12 most cited of 170 All 170 (JSON)
- 2026CLoSeR: Closing the Loop for Long-Context Streaming Reconstruction1 Oct 2026
- 2026OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Compute28 Sep 2026
- 2026VGGT-GS SLAM: Uncalibrated Monocular Gaussian Splatting SLAM with Feed-Forward Priors17 Sep 2026
- 2026DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers11 Sep 2026
- 2026UniSim-SLAM: Feed-Forward SLAM with Unified Sim(3) Optimization3 Aug 2026
- 2026Planar-SfM: Camera Pose Estimation via Homography Graph Embeddings30 Jun 2026
- 2026MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry15 Jun 2026
- 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application2 Jun 2026
- 2026Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence19 May 2026
- 2026VGGT-Omega14 May 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM24 Apr 2026
12 newest of 170 All 170 (JSON)