DeMoN: Depth and Motion Network for Learning Monocular Stereo
Not on the map. Why?
Abstract (arXiv)
In this paper we formulate structure from motion as a learning problem. We train a convolutional network end-to-end to compute depth and camera motion from successive, unconstrained image pairs. The architecture is composed of multiple stacked encoder-decoder networks, the core part being an iterative network that is able to improve its own predictions. The network estimates not only depth and motion, but additionally surface normals, optical flow between the images and confidence of the matching. A crucial component of the approach is a training loss based on spatial relative differences. Compared to traditional two-frame structure from motion methods, results are more accurate and more robust. In contrast to the popular depth-from-single-image networks, DeMoN learns the concept of matching and, thus, better generalizes to structures not seen during training.
Key facts. Cited by 200 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Dec 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 52; 2022: 39; 2023: 38; 2024: 33; 2025: 27; 2026: 11.
200 papers.
Sort:
By area:
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
- 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
- 2022Unifying Flow, Stereo and Depth Estimationcited by 270
- 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
- 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
- 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
38 more
- 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
- 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
- 2023NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAMcited by 162
- 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 146
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
- 2023NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidancecited by 137
- 2022Map-free Visual Relocalization: Metric Pose Relative to a Single Imagecited by 112
- 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 107
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
- 2025Cameras as Relative Positional Encodingcited by 94
- 2024Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizercited by 73
- 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
- 2021Monocular 3D Object Detection: An Extrinsic Parameter Free Approachcited by 69
- 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 69
- 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
- 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
- 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
- 2024LightIt: Illumination Modeling and Control for Diffusion Modelscited by 56
- 2024VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstructioncited by 50
- 2021Wide-Baseline Relative Camera Pose Estimation with Directional Learningcited by 49
- 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
- 2021MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutionscited by 43
- 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 43
- 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Imagecited by 42
- 2023Dense RGB SLAM with Neural Implicit Mapscited by 42
- 2021Planar Surface Reconstruction from Sparse Viewscited by 40
- 2021Deep Two-View Structure-from-Motion Revisitedcited by 39
- 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 39
- 2021MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 38
- 2022A Benchmark and a Baseline for Robust Multi-view Depth Estimationcited by 38
- 2023NOPE: Novel Object Pose Estimation from a Single Imagecited by 38
- 2021Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervisioncited by 37
- 2025Detect Anything 3D in the Wildcited by 32
- 2021Projective Manifold Gradient Layer for Deep Rotation Regressioncited by 31
- 2021Input-level Inductive Biases for 3D Reconstructioncited by 31
- 2024GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstructioncited by 31
- 2023PMatch: Paired Masked Image Modeling for Dense Geometric Matchingcited by 29
- 2023Deep Event Visual Odometrycited by 29
50 most cited of 200 All 200 (JSON)
- 2026Reconstructing the Dynamic World: A Representation-Centric View of 4D Scene Reconstruction30 Sep 2026
- 2026DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers11 Sep 2026
- 2026SLAMFormer-∞: Infinite SLAM Transformer for Unbounded Frontend and Backend Processing4 Aug 2026
- 2026Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence19 May 2026
- 2026VGGT-Omega14 May 2026
- 2026Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM24 Apr 2026
- 2026URoPE: Universal Relative Position Embedding across Geometric Spaces20 Apr 2026
- 2026VGGT-SLAM++8 Apr 2026
- 2026UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perception26 Feb 2026
- 2026RayRoPE: Projective Ray Positional Encoding for Multi-view Attention21 Jan 2026
- 2026Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning20 Jan 2026
- 2025A Study of Finetuning Video Transformers for Multi-view Geometry Tasks21 Dec 2025
38 more
- 2025G3Splat: Geometrically Consistent Generalizable Gaussian Splatting19 Dec 2025
- 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend25 Nov 2025
- 2025DetAny4D: Detect Anything 4D Temporally in a Streaming RGB Video24 Nov 2025
- 2025Lumos3D: A Single-Forward Framework for Low-Light 3D Scene Restoration12 Nov 2025
- 2025MoRe: Monocular Geometry Refinement via Graph Optimization for Cross-View Consistency8 Oct 2025
- 2025Stylos: Multi-View 3D Stylization with Single-Forward Gaussian Splatting30 Sep 2025
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstruction16 Sep 2025
- 2025DOMR: Establishing Cross-View Segmentation via Dense Object Matching6 Aug 2025
- 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences22 Jul 2025
- 2025An Evaluation of DUSt3R/MASt3R/VGGT 3D Reconstruction on Photogrammetric Aerial Blocks20 Jul 2025
- 2025Cameras as Relative Positional Encoding14 Jul 2025
- 2025PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models11 Jul 2025
- 2025Test3R: Learning to Reconstruct 3D at Test Time16 Jun 2025
- 2025DELTA: Dense Depth from Events and LiDAR using Transformer's Attention5 May 2025
- 2025Efficient 2D to Full 3D Human Pose Uplifting including Joint Rotations14 Apr 2025
- 2025Detect Anything 3D in the Wild10 Apr 2025
- 2025MVSAnywhere: Zero-Shot Multi-View Stereo28 Mar 2025
- 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors21 Mar 2025
- 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction20 Mar 2025
- 2025VGGT: Visual Geometry Grounded Transformer14 Mar 2025
- 2025SIRE: SE(3) Intrinsic Rigidity Embeddings10 Mar 2025
- 2025PE3R: Perception-Efficient 3D Reconstruction10 Mar 2025
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstruction3 Mar 2025
- 2025Matrix3D: Large Photogrammetry Model All-in-One11 Feb 2025
- 2025Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion30 Jan 2025
- 2025JOG3R: Towards 3D-Consistent Video Generators2 Jan 2025
- 2024PhysDepth: Plug-and-Play Physical Refinement for Monocular Depth Estimation in Challenging Environments5 Dec 2024
- 2024World-consistent Video Diffusion with Explicit 3D Modeling2 Dec 2024
- 2024MBA-SLAM: Motion Blur Aware Gaussian Splatting SLAM13 Nov 2024
- 2024SE(3) Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation11 Nov 2024
- 20243D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction1 Nov 2024
- 2024LGU-SLAM: Learnable Gaussian Uncertainty Matching with Deformable Correlation Sampling for Deep Visual SLAM30 Oct 2024
- 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3D24 Oct 2024
- 2024DepthSplat: Connecting Gaussian Splatting and Depth17 Oct 2024
- 2024Refinement of Monocular Depth Maps via Multi-View Differentiable Rendering4 Oct 2024
- 2024Underground Mapping and Localization Based on Ground-Penetrating Radar24 Sep 2024
- 2024Alignist: CAD-Informed Orientation Distribution Estimation by Fusing Shape and Correspondences10 Sep 2024
- 2024Str-L Pose: Integrating Point and Structured Line for Relative Pose Estimation in Dual-Graph28 Aug 2024
50 newest of 200 All 200 (JSON)
- 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
- 2024LightIt: Illumination Modeling and Control for Diffusion Modelscited by 56
- 2023Diffusion Model for Dense Matchingcited by 21
- 2022DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Modelscited by 12
- 2024World-consistent Video Diffusion with Explicit 3D Modeling2 Dec 2024
- 2024LightIt: Illumination Modeling and Control for Diffusion Models15 Mar 2024
- 2023Diffusion Model for Dense Matching30 May 2023
- 2022DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models17 Dec 2022
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
- 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
- 2022Unifying Flow, Stereo and Depth Estimationcited by 270
- 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
- 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
- 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
12 most cited of 196 All 196 (JSON)
- 2026Reconstructing the Dynamic World: A Representation-Centric View of 4D Scene Reconstruction30 Sep 2026
- 2026DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers11 Sep 2026
- 2026SLAMFormer-∞: Infinite SLAM Transformer for Unbounded Frontend and Backend Processing4 Aug 2026
- 2026Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence19 May 2026
- 2026VGGT-Omega14 May 2026
- 2026Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM24 Apr 2026
- 2026URoPE: Universal Relative Position Embedding across Geometric Spaces20 Apr 2026
- 2026VGGT-SLAM++8 Apr 2026
- 2026UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perception26 Feb 2026
- 2026RayRoPE: Projective Ray Positional Encoding for Multi-view Attention21 Jan 2026
- 2026Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning20 Jan 2026
- 2025A Study of Finetuning Video Transformers for Multi-view Geometry Tasks21 Dec 2025
12 newest of 196 All 196 (JSON)