mlsanityData to 7 Oct 2026

Self-Supervised Monocular Scene Decomposition and Depth Estimation

arXiv 2110.11275 ↗First posted 21 Oct 2021cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Self-supervised monocular depth estimation approaches either ignore independently moving objects in the scene or need a separate segmentation step to identify them. We propose MonoDepthSeg to jointly estimate depth and segment moving objects from monocular video without using any ground-truth labels. We decompose the scene into a fixed number of components where each component corresponds to a region on the image with its own transformation matrix representing its motion. We estimate both the mask and the motion of each component efficiently with a shared encoder. We evaluate our method on three driving datasets and show that our model clearly improves depth estimation while decomposing the scene into separately moving components.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 21 Oct 2021.

Builds on

29 of 44 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  7. 2015Spatial Transformer Networkscited by 1,423
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  10. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  11. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  12. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
17 more
  1. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  2. 2017Sparsity Invariant CNNscited by 325
  3. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  4. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  5. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  6. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  7. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  8. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  9. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  10. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  11. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  12. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  13. 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
  14. 2016SE3-Nets: Learning Rigid Body Motion using Deep Neural Networkscited by 48
  15. 2020Unsupervised part representation by Flow Capsulescited by 25
  16. 2019Learning Independent Object Motion from Unlabelled Stereoscopic Videoscited by 17
  17. 2016Semantic 3D Reconstruction with Continuous Regularization and Ray Potentials Using a Visibility Consistency Constraintcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3.

8 papers.

  1. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  2. 2023Multi-Object Discovery by Low-Dimensional Object Motioncited by 16
  3. 2022Stochastic Video Prediction with Structure and Motioncited by 8
  4. 2022Instance-aware multi-object self-supervision for monocular depth predictioncited by 5
  5. 2022Dyna-DM: Dynamic Object-aware Self-supervised Monocular Depth Mapscited by 3
  6. 2023STDepthFormer: Predicting Spatio-temporal Depth from Video with a Self-supervised Transformer Modelcited by 1
  7. 2022Stochastic Future Prediction in Real World Driving Scenarioscited by 0
  8. 2023DepthP+P: Metric Accurate Monocular Depth Estimation using Planar and Parallaxcited by 0