mlsanityData to 14 Oct 2026

MGNiceNet: Unified Monocular Geometric Scene Understanding

arXiv 2411.11466 ↗First posted 18 Nov 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Monocular geometric scene understanding combines panoptic segmentation and self-supervised depth estimation, focusing on real-time application in autonomous vehicles. We introduce MGNiceNet, a unified approach that uses a linked kernel formulation for panoptic segmentation and self-supervised depth estimation. MGNiceNet is based on the state-of-the-art real-time panoptic segmentation method RT-K-Net and extends the architecture to cover both panoptic segmentation and self-supervised monocular depth estimation. To this end, we introduce a tightly coupled self-supervised depth estimation predictor that explicitly uses information from the panoptic path for depth prediction. Furthermore, we introduce a panoptic-guided motion masking method to improve depth estimation without relying on video panoptic segmentation annotations. We evaluate our method on two popular autonomous driving datasets, Cityscapes and KITTI. Our model shows state-of-the-art results compared to other real-time methods and closes the gap to computationally more demanding methods. Source code and trained models are available at https://github.com/markusschoen/MGNiceNet.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

44 of 60 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  6. 2022A ConvNet for the 2020scited by 3,579
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  10. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  11. 2018Panoptic Segmentationcited by 802
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
32 more
  1. 2019Panoptic Feature Pyramid Networkscited by 645
  2. 2017Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentationscited by 620
  3. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  4. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  5. 2019Panoptic-DeepLabcited by 363
  6. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  7. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  8. 2017Sparsity Invariant CNNscited by 326
  9. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  10. 2021K-Net: Towards Unified Image Segmentationcited by 212
  11. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  12. 2015ParseNet: Looking Wider to See Bettercited by 177
  13. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  14. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  15. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  16. 2019Seamless Scene Segmentationcited by 112
  17. 2020EfficientPS: Efficient Panoptic Segmentationcited by 111
  18. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 96
  19. 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentationcited by 60
  20. 2020SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Drivingcited by 48
  21. 2020Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applicationscited by 40
  22. 2021PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentationcited by 36
  23. 2020Scaling Wide Residual Networks for Panoptic Segmentationcited by 32
  24. 2019Real-Time Panoptic Segmentation from Dense Detectionscited by 31
  25. 2022RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformercited by 31
  26. 2022MGNet: Monocular Geometric Scene Understanding for Autonomous Drivingcited by 27
  27. 2023SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimationcited by 21
  28. 2022PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentationcited by 14
  29. 2022UniDAformer: Unified Domain Adaptive Panoptic Segmentation Transformer via Hierarchical Mask Calibrationcited by 8
  30. 2022MonoDVPS: A Self-Supervised Monocular Depth Estimation Approach to Depth-aware Video Panoptic Segmentationcited by 7
  31. 2023Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learningcited by 6
  32. 2023RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentationcited by 1

Built on by

Not cited yet.