mlsanityData to 7 Oct 2026

MGNet: Monocular Geometric Scene Understanding for Autonomous Driving

arXiv 2206.13199 ↗First posted 27 Jun 2022cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

We introduce MGNet, a multi-task framework for monocular geometric scene understanding. We define monocular geometric scene understanding as the combination of two known tasks: Panoptic segmentation and self-supervised monocular depth estimation. Panoptic segmentation captures the full scene not only semantically, but also on an instance basis. Self-supervised monocular depth estimation uses geometric constraints derived from the camera measurement model in order to measure depth from monocular video sequences only. To the best of our knowledge, we are the first to propose the combination of these two tasks in one single model. Our model is designed with focus on low latency to provide fast inference in real-time on a single consumer-grade GPU. During deployment, our model produces dense 3D point clouds with instance aware semantic labels from single high-resolution camera images. We evaluate our model on two popular autonomous driving benchmarks, i.e., Cityscapes and KITTI, and show competitive performance among other real-time capable methods. Source code is available at https://github.com/markusschoen/MGNet.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 68 of its 85 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

68 of 85 references.

References · 68

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2017Mask R-CNNcited by 7,259
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  9. 2016Pyramid Scene Parsing Networkcited by 2,472
  10. 2019Searching for MobileNetV3cited by 1,984
  11. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  12. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
38 more
  1. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  2. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  3. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  4. 2018Panoptic Segmentationcited by 796
  5. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  6. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  7. 2019Panoptic Feature Pyramid Networkscited by 644
  8. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  9. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  10. 2019Panoptic-DeepLabcited by 362
  11. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  12. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  13. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  14. 2017Sparsity Invariant CNNscited by 325
  15. 2019SOLO: Segmenting Objects by Locationscited by 289
  16. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  17. 2020DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolutioncited by 222
  18. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
  19. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  20. 2015ParseNet: Looking Wider to See Bettercited by 176
  21. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  22. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  23. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  24. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  25. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  26. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  27. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  28. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  29. 2018Improving Semantic Segmentation via Video Propagation and Label Relaxationcited by 134
  30. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  31. 2019Seamless Scene Segmentationcited by 112
  32. 2020EfficientPS: Efficient Panoptic Segmentationcited by 111
  33. 2020Robust Learning Through Cross-Task Consistencycited by 105
  34. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  35. 2017UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learningcited by 95
  36. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 95
  37. 2019PolyTransform: Deep Polygon Transformer for Instance Segmentationcited by 92
  38. 2020Fully Convolutional Networks for Panoptic Segmentationcited by 92

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 8; 2025: 10; 2026: 1.

27 papers.

Sort:

By area:

  1. 2024TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detectioncited by 21
  2. 2024PanDA: Towards Panoramic Depth Anything with Unlabeled Panoramas and Mobius Spatial Augmentationcited by 21
  3. 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 18
  4. 2024ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimationcited by 15
  5. 2023The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimationcited by 13
  6. 2024SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Imagescited by 13
  7. 2022JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenescited by 11
  8. 2023Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Drivingcited by 8
  9. 2023Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learningcited by 6
  10. 2023EC-Depth: Exploring the consistency of self-supervised monocular depth estimation in challenging scenescited by 4
  11. 2024Panoptic Perception for Autonomous Driving: A Surveycited by 4
  12. 2025Always Clear Depth: Robust Monocular Depth Estimation under Adverse Weathercited by 4
15 more
  1. 2025Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cuescited by 3
  2. 2025M2H: Multi-Task Learning with Efficient Window-Based Cross-Task Attention for Monocular Spatial Perceptioncited by 3
  3. 2022PanDepth: Joint Panoptic Segmentation and Depth Completioncited by 2
  4. 2023A Dynamic Feature Interaction Framework for Multi-task Visual Perceptioncited by 2
  5. 2025ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimationcited by 2
  6. 2023RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentationcited by 1
  7. 2024CRF360D: Monocular 360 Depth Estimation via Spherical Fully-Connected CRFscited by 1
  8. 2024FusionSAM: Visual Multi-Modal Learning with Segment Anythingcited by 1
  9. 2025Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attackscited by 1
  10. 2024MGNiceNet: Unified Monocular Geometric Scene Understandingcited by 0
  11. 2025Depth as Points: Center Point-based Depth Estimationcited by 0
  12. 2025LMDepth: Lightweight Mamba-based Monocular Depth Estimation for Real-World Deploymentcited by 0
  13. 2025VGLD: Visually-Guided Linguistic Disambiguation for Monocular Depth Scale Recoverycited by 0
  14. 2025BadDepth: Backdoor Attacks Against Monocular Depth Estimation in the Physical Worldcited by 0
  15. 2026LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Predictioncited by 0