mlsanityData to 7 Oct 2026

SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular images

arXiv 2101.07422 ↗First posted 19 Jan 2021cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

Depth estimation and semantic segmentation play essential roles in scene understanding. The state-of-the-art methods employ multi-task learning to simultaneously learn models for these two tasks at the pixel-wise level. They usually focus on sharing the common features or stitching feature maps from the corresponding branches. However, these methods lack in-depth consideration on the correlation of the geometric cues and the scene parsing. In this paper, we first introduce the concept of semantic objectness to exploit the geometric relationship of these two tasks through an analysis of the imaging process, then propose a Semantic Object Segmentation and Depth Estimation Network (SOSD-Net) based on the objectness assumption. To the best of our knowledge, SOSD-Net is the first network that exploits the geometry constraint for simultaneous monocular depth estimation and semantic segmentation. In addition, considering the mutual implicit relationship between these two tasks, we exploit the iterative idea from the expectation-maximization algorithm to train the proposed network more effectively. Extensive experimental results on the Cityscapes and NYU v2 dataset are presented to demonstrate the superior performance of the proposed approach.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 45 of its 67 references that are in the map. First posted on arXiv 19 Jan 2021.

Builds on

45 of 67 references.

References · 45

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  6. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  7. 2016Pyramid Scene Parsing Networkcited by 2,472
  8. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  9. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  10. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  11. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  12. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
33 more
  1. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  2. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  3. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  4. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  5. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  6. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  7. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  8. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
  9. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  10. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  11. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 463
  12. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  13. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  14. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  15. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  16. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  17. 2015Conditional Random Fields as Recurrent Neural Networkscited by 213
  18. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  19. 2015ParseNet: Looking Wider to See Bettercited by 176
  20. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  21. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 124
  22. 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
  23. 2015Efficient piecewise training of deep structured models for semantic segmentationcited by 71
  24. 2016Full-Resolution Residual Networks for Semantic Segmentation in Street Scenescited by 71
  25. 2017Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimationcited by 70
  26. 2019DepthTransfer: Depth Extraction from Video Using Non-parametric Samplingcited by 60
  27. 2015Semantic Image Segmentation via Deep Parsing Networkcited by 44
  28. 2016Joint Semantic Segmentation and Depth Estimation with Deep Convolutional Networkscited by 36
  29. 2018Learning Depth from Single Images with Deep Neural Network Embedding Focal Lengthcited by 25
  30. 2020Deep Feature Augmentation for Occluded Image Classificationcited by 21
  31. 2018Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Networkcited by 20
  32. 2019MDFN: Multi-Scale Deep Feature Learning Network for Object Detectioncited by 19
  33. 2017Analyzing Modular CNN Architectures for Joint Depth Prediction and Semantic Segmentationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 7; 2023: 4; 2024: 8; 2025: 2; 2026: 1.

33 papers.

Sort:

By area:

  1. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  2. 2021BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimationcited by 31
  3. 2023Semantic Image Segmentation: Two Decades of Researchcited by 26
  4. 2023EVEN: An Event-Based Framework for Monocular Depth Estimation at Adverse Night Conditionscited by 19
  5. 2024Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Surveycited by 15
  6. 2021GCNDepth: Self-supervised Monocular Depth Estimation based on Graph Convolutional Networkcited by 12
  7. 2021Few-Shot Learning by Integrating Spatial and Frequency Representationcited by 11
  8. 2021An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Trainingcited by 10
  9. 2022Aggregating Global Features into Local Vision Transformercited by 10
  10. 2021A Domain Gap Aware Generative Adversarial Network for Multi-domain Image Translationcited by 8
  11. 2023Joint Depth Prediction and Semantic Segmentation with Multi-View SAMcited by 8
  12. 2021SUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised Monocular Depth Estimationcited by 7
21 more
  1. 2022Dynamic Label Assignment for Object Detection by Combining Predicted IoUs and Anchor IoUscited by 6
  2. 2023On the Real-Time Semantic Segmentation of Aphid Clusters in the Wildcited by 6
  3. 2024Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimationcited by 6
  4. 2022DepthFake: a depth-based strategy for detecting Deepfake videoscited by 5
  5. 2022SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completioncited by 5
  6. 2021A Discriminative Channel Diversification Network for Image Classificationcited by 4
  7. 2024A New Dataset and Comparative Study for Aphid Cluster Detection and Segmentation in Sorghum Fieldscited by 3
  8. 2021CI-Net: Contextual Information for Joint Semantic Segmentation and Depth Estimationcited by 2
  9. 2021Multiple Classifiers Based Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 2
  10. 2022PanDepth: Joint Panoptic Segmentation and Depth Completioncited by 2
  11. 2024Kitchen Food Waste Image Segmentation and Classification for Compost Nutrients Estimationcited by 2
  12. 2024Multi-Layer Dense Attention Decoder for Polyp Segmentationcited by 2
  13. 2024A Comparative Study on Multi-task Uncertainty Quantification in Semantic Segmentation and Monocular Depth Estimationcited by 2
  14. 2024Vision-Driven 2D Supervised Fine-Tuning Framework for Bird's Eye View Perceptioncited by 2
  15. 2021Simpler Does It: Generating Semantic Labels with Objectness Guidancecited by 1
  16. 2022TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistencycited by 1
  17. 2022Hybrid Transformer Based Feature Fusion for Self-Supervised Monocular Depth Estimationcited by 1
  18. 2025DenseFormer: Learning Dense Depth Map from Sparse Depth and Image via Conditional Diffusion Modelcited by 1
  19. 2021Training Deep Neural Networks via Branch-and-Boundcited by 0
  20. 2025Co-SemDepth: Fast Joint Semantic Segmentation and Depth Estimation on Aerial Imagescited by 0
  21. 2026Efficient RGB-D Scene Understanding via Multi-task Adaptive Learning and Cross-dimensional Feature Guidancecited by 0