mlsanityData to 14 Oct 2026

Unsupervised Learning of Depth, Camera Pose and Optical Flow from Monocular Video

arXiv 2205.09821 ↗First posted 19 May 2022cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose DFPNet -- an unsupervised, joint learning system for monocular Depth, Optical Flow and egomotion (Camera Pose) estimation from monocular image sequences. Due to the nature of 3D scene geometry these three components are coupled. We leverage this fact to jointly train all the three components in an end-to-end manner. A single composite loss function -- which involves image reconstruction-based loss for depth & optical flow, bidirectional consistency checks and smoothness loss components -- is used to train the network. Using hyperparameter tuning, we are able to reduce the model size to less than 5% (8.4M parameters) of state-of-the-art DFP models. Evaluation on KITTI and Cityscapes driving datasets reveals that our model achieves results comparable to state-of-the-art in all of the three tasks, even with the significantly smaller model size.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

24 of 37 references.

References · 24

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  3. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  4. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  6. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  7. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  8. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  9. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  10. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 360
  11. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  12. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
12 more
  1. 2017Geometry-Aware Learning of Maps for Camera Localizationcited by 161
  2. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  3. 2020What Matters in Unsupervised Optical Flowcited by 124
  4. 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
  5. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  6. 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
  7. 2019Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semanticscited by 49
  8. 2018Unsupervised Adversarial Depth Estimation using Cycled Generative Networkscited by 45
  9. 2016Optical Flow with Semantic Segmentation and Localized Layerscited by 21
  10. 2017Optical Flow in Mostly Rigid Scenescited by 20
  11. 2017Fast Multi-frame Stereo Scene Flow with Motion Segmentationcited by 6
  12. 2017Topometric Localization with Deep Learningcited by 4

Built on by

Not cited yet.