mlsanityData to 14 Oct 2026

CI-Net: Contextual Information for Joint Semantic Segmentation and Depth Estimation

arXiv 2107.13800 ↗First posted 29 Jul 2021cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Monocular depth estimation and semantic segmentation are two fundamental goals of scene understanding. Due to the advantages of task interaction, many works study the joint task learning algorithm. However, most existing methods fail to fully leverage the semantic labels, ignoring the provided context structures and only using them to supervise the prediction of segmentation split, which limit the performance of both tasks. In this paper, we propose a network injected with contextual information (CI-Net) to solve the problem. Specifically, we introduce self-attention block in the encoder to generate attention map. With supervision from the ideal attention map created by semantic label, the network is embedded with contextual information so that it could understand scene better and utilize correlated features to make accurate prediction. Besides, a feature sharing module is constructed to make the task-specific features deeply fused and a consistency loss is devised to make the features mutually guided. We evaluate the proposed CI-Net on the NYU-Depth-v2 and SUN-RGBD datasets. The experimental results validate that our proposed CI-Net could effectively improve the accuracy of semantic segmentation and depth estimation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 47 of its 73 references that are in the map. First posted on arXiv 29 Jul 2021.

Builds on

47 of 73 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  3. 2017Squeeze-and-Excitation Networkscited by 4,789
  4. 2017Non-local Neural Networkscited by 2,293
  5. 2015Spatial Transformer Networkscited by 1,429
  6. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  8. 2018Self-Attention with Relative Position Representationscited by 1,120
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  10. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  11. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  12. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 554
35 more
  1. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  2. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 465
  3. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  4. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  5. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  6. 2017Dilated Residual Networkscited by 294
  7. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 234
  8. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  9. 2015Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understandingcited by 218
  10. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  11. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  12. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 145
  13. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 142
  14. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  15. 2020Context Prior for Scene Segmentationcited by 121
  16. 2019ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentationcited by 107
  17. 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
  18. 2016Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networkscited by 91
  19. 2020Guiding Monocular Depth Estimation Using Depth-Attention Volumecited by 82
  20. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  21. 2020Temporally Distributed Networks for Fast Video Semantic Segmentationcited by 77
  22. 2018Depth-aware CNN for RGB-D Segmentationcited by 74
  23. 2018Self-Supervised Model Adaptation for Multimodal Semantic Segmentationcited by 73
  24. 2015Efficient piecewise training of deep structured models for semantic segmentationcited by 71
  25. 2017Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimationcited by 70
  26. 2019SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimationcited by 59
  27. 2020Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentationcited by 48
  28. 2018LEGO: Learning Edge with Geometry all at Once by Watching Videoscited by 47
  29. 2021SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular imagescited by 33
  30. 2018Light-Weight RefineNet for Real-Time Semantic Segmentationcited by 27
  31. 2017Monocular Depth Estimation with Hierarchical Fusion of Dilated CNNs and Soft-Weighted-Sum Inferencecited by 24
  32. 2019Attention-based Context Aggregation Network for Monocular Depth Estimationcited by 18
  33. 2020Improving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasetscited by 14
  34. 2019Deep Learning and Control Algorithms of Direct Perception for Autonomous Drivingcited by 11
  35. 2018Deep multi-scale architectures for monocular depth estimationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023Semantic Image Segmentation: Two Decades of Researchcited by 27
  2. 2026Mitigating Privacy Risk via Forget Set-Free Unlearningcited by 2