GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Not on the map. Why?
Abstract (arXiv)
We propose GeoNet, a jointly unsupervised learning framework for monocular depth, optical flow and ego-motion estimation from videos. The three components are coupled by the nature of 3D scene geometry, jointly learned by our framework in an end-to-end manner. Specifically, geometric relationships are extracted over the predictions of individual modules and then combined as an image reconstruction loss, reasoning about static and dynamic scene parts separately. Furthermore, we propose an adaptive geometric consistency loss to increase robustness towards outliers and non-Lambertian regions, which resolves occlusions and texture ambiguities effectively. Experimentation on the KITTI driving dataset reveals that our scheme achieves state-of-the-art results in all of the three tasks, performing better than previously unsupervised methods and comparably with supervised ones.
Key facts. Cited by 342 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Mar 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 90; 2022: 91; 2023: 73; 2024: 42; 2025: 30; 2026: 16.
342 papers.
Sort:
By area:
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
- 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
- 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
- 2023Robust Dynamic Radiance Fieldscited by 194
- 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
- 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
- 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 178
38 more
- 2022ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformercited by 171
- 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
- 2022P3Depth: Monocular Depth Estimation with a Piecewise Planarity Priorcited by 107
- 2023Progressively Optimized Local Radiance Fields for Robust View Synthesiscited by 103
- 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
- 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
- 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 92
- 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
- 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
- 2021Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescuecited by 72
- 2024Deep Patch Visual SLAMcited by 71
- 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
- 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
- 2021Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimationcited by 63
- 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
- 2021Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Darkcited by 58
- 2021OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Drivingcited by 57
- 2021Consistent Depth of Moving Objects in Videocited by 56
- 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 56
- 2022TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baselinecited by 55
- 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillationcited by 55
- 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 52
- 2021Wide-Baseline Relative Camera Pose Estimation with Directional Learningcited by 49
- 2021Excavating the Potential Capacity of Self-Supervised Monocular Depth Estimationcited by 48
- 2022Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depthcited by 48
- 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 47
- 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
- 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Imagecited by 42
- 2024SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splattingcited by 42
- 2022textttGradICON: Approximate Diffeomorphisms via Gradient Inverse Consistencycited by 41
- 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 41
- 2021Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completioncited by 39
- 2021Deep Two-View Structure-from-Motion Revisitedcited by 39
- 2021Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walkcited by 39
- 2021TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereocited by 39
- 2023FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flowcited by 39
- 2023R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Camerascited by 39
- 2021MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 38
50 most cited of 342 All 342 (JSON)
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportation22 Sep 2026
- 2026CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation4 Sep 2026
- 2026Weather-Conditioned Depth Anything4 Sep 2026
- 2026Monocular Depth Estimation from a Single Image: Progress and Opportunities1 Sep 2026
- 2026An active-learning framework for real-time depth perception from monocular vision streams5 Aug 2026
- 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device9 Jul 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions8 May 2026
- 2026SS3D: End2End Self-Supervised 3D from Web Videos24 Apr 2026
- 2026PoseFM: Relative Camera Pose Estimation Through Flow Matching24 Apr 2026
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2026Adaptive Depth-converted-Scale Convolution for Self-supervised Monocular Depth Estimation9 Apr 2026
38 more
- 2026OpenVO: Open-World Visual Odometry with Temporal Dynamics Awareness22 Feb 2026
- 2026PuriLight: A Lightweight Shuffle and Purification Framework for Monocular Depth Estimation11 Feb 2026
- 2026SHED Light on Segmentation for Dense Prediction30 Jan 2026
- 2026studentSplat: Your Student Model Learns Single-view 3D Gaussian Splatting16 Jan 2026
- 2025Visual Autoregressive Modelling for Monocular Depth Estimation27 Dec 2025
- 2025DASP: Self-supervised Nighttime Monocular Depth Estimation with Domain Adaptation of Spatiotemporal Priors16 Dec 2025
- 2025Audio-Visual Camera Pose Estimation with Passive Scene Sounds and In-the-Wild Video13 Dec 2025
- 2025CylinderDepth: Cylindrical Spatial Attention for Multi-View Consistent Self-Supervised Surround Depth Estimation20 Nov 2025
- 2025RadarMP: Motion Perception for 4D mmWave Radar in Autonomous Driving15 Nov 2025
- 2025True Self-Supervised Novel View Synthesis is Transferable15 Oct 2025
- 2025DrivingScene: A Multi-Task Online Feed-Forward 3D Gaussian Splatting Method for Dynamic Driving Scenes14 Oct 2025
- 2025Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimation10 Oct 2025
- 2025UM-Depth : Uncertainty Masked Self-Supervised Monocular Depth Estimation with Visual Odometry17 Sep 2025
- 2025CoProU-VO: Combining Projected Uncertainty for End-to-End Unsupervised Monocular Visual Odometry1 Aug 2025
- 2025Reconstructing 4D Spatial Intelligence: A Survey28 Jul 2025
- 2025Endoscopic Depth Estimation Based on Deep Learning: A Survey28 Jul 2025
- 2025VOCAL: Visual Odometry via ContrAstive Learning30 Jun 2025
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation25 Jun 2025
- 2025GS-2DGS: Geometrically Supervised 2DGS for Reflective Object Reconstruction16 Jun 2025
- 2025ZeroVO: Visual Odometry with Minimal Assumptions9 Jun 2025
- 2025Toward Better SSIM Loss for Unsupervised Monocular Depth Estimation5 Jun 2025
- 2025JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splatting4 Jun 2025
- 2025Dynamic Camera Poses and Where to Find Them24 Apr 2025
- 2025The Fourth Monocular Depth Estimation Challenge24 Apr 2025
- 2025An Online Adaptation Method for Robust Depth Estimation and Visual Odometry in the Open World16 Apr 2025
- 2025D2USt3R: Enhancing 3D Reconstruction for Dynamic Scenes8 Apr 2025
- 2025OmniCam: Unified Multimodal Video Generation via Camera Control3 Apr 2025
- 2025AlignDiff: Learning Physically-Grounded Camera Alignment via Diffusion27 Mar 2025
- 2025Can Video Diffusion Model Reconstruct 4D Geometry?27 Mar 2025
- 2025FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields15 Mar 2025
- 2025Leveraging Motion Information for Better Self-Supervised Video Correspondence Learning15 Mar 2025
- 2025Project-and-Fuse: Improving RGB-D Semantic Segmentation via Graph Convolution Networks31 Jan 2025
- 2025Zero-Shot Monocular Scene Flow Estimation in the Wild17 Jan 2025
- 2025JOG3R: Towards 3D-Consistent Video Generators2 Jan 2025
- 2024ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction15 Dec 2024
- 2024On-Device Self-Supervised Learning of Low-Latency Monocular Depth from Only Events9 Dec 2024
- 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail5 Dec 2024
- 2024AVS-Net: Audio-Visual Scale Net for Self-supervised Monocular Metric Depth Estimation2 Dec 2024
50 newest of 342 All 342 (JSON)
- 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditionscited by 29
- 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Modelcited by 13
- 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimationcited by 6
- 2026PoseFM: Relative Camera Pose Estimation Through Flow Matchingcited by 1
- 2024EDADepth: Enhanced Data Augmentation for Monocular Depth Estimationcited by 0
- 2026PoseFM: Relative Camera Pose Estimation Through Flow Matching24 Apr 2026
- 2025StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation25 Jun 2025
- 2025Can Video Diffusion Model Reconstruct 4D Geometry?27 Mar 2025
- 2024EDADepth: Enhanced Data Augmentation for Monocular Depth Estimation10 Sep 2024
- 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions23 Jul 2024
- 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model13 Nov 2023
- 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillationcited by 55
- 2021Excavating the Potential Capacity of Self-Supervised Monocular Depth Estimationcited by 48
- 2021X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillationcited by 12
- 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimationcited by 6
- 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillation10 Apr 2022
- 2021X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillation24 Oct 2021
- 2021Excavating the Potential Capacity of Self-Supervised Monocular Depth Estimation26 Sep 2021
- 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation8 Jun 2021
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
- 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
- 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
- 2023Robust Dynamic Radiance Fieldscited by 194
- 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
- 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
- 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 178
12 most cited of 328 All 328 (JSON)
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportation22 Sep 2026
- 2026CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation4 Sep 2026
- 2026Weather-Conditioned Depth Anything4 Sep 2026
- 2026Monocular Depth Estimation from a Single Image: Progress and Opportunities1 Sep 2026
- 2026An active-learning framework for real-time depth perception from monocular vision streams5 Aug 2026
- 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device9 Jul 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions8 May 2026
- 2026SS3D: End2End Self-Supervised 3D from Web Videos24 Apr 2026
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2026Adaptive Depth-converted-Scale Convolution for Self-supervised Monocular Depth Estimation9 Apr 2026
- 2026OpenVO: Open-World Visual Odometry with Temporal Dynamics Awareness22 Feb 2026
12 newest of 328 All 328 (JSON)