PanDepth: Joint Panoptic Segmentation and Depth Completion
Not on the map. Why?
Abstract (arXiv)
Understanding 3D environments semantically is pivotal in autonomous driving applications where multiple computer vision tasks are involved. Multi-task models provide different types of outputs for a given scene, yielding a more holistic representation while keeping the computational cost low. We propose a multi-task model for panoptic segmentation and depth completion using RGB images and sparse depth maps. Our model successfully predicts fully dense depth maps and performs semantic segmentation, instance segmentation, and panoptic segmentation for every input frame. Extensive experiments were done on the Virtual KITTI 2 dataset and we demonstrate that our model solves multiple tasks, without a significant increase in computational cost, while keeping high accuracy performance. Code is available at https://github.com/juanb09111/PanDepth.git
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 46 of its 53 references that are in the map. First posted on arXiv 29 Dec 2022.
Builds on
46 of 53 references.
References · 46
- 2017Attention Is All You Needcited by 60,361
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2017Mask R-CNNcited by 7,277
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
34 more
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
- 2018Panoptic Segmentationcited by 802
- 2019Panoptic Feature Pyramid Networkscited by 645
- 2018End-to-End Multi-Task Learning with Attentioncited by 534
- 2019Panoptic-DeepLabcited by 363
- 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
- 2020Virtual KITTI 2cited by 353
- 2017Sparsity Invariant CNNscited by 326
- 2020Non-Local Spatial Propagation Network for Depth Completioncited by 195
- 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
- 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
- 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
- 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
- 2020Learning to Branch for Multi-Task Learningcited by 115
- 2019Learning Guided Convolutional Network for Depth Completioncited by 111
- 2020EfficientPS: Efficient Panoptic Segmentationcited by 111
- 2018Auxiliary Tasks in Multi-task Learningcited by 108
- 2017In-Place Activated BatchNorm for Memory-Optimized Training of DNNscited by 81
- 2019Sparse and noisy LiDAR completion with RGB guidance and uncertaintycited by 79
- 2018Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentationcited by 76
- 2018Attention-guided Unified Network for Panoptic Segmentationcited by 71
- 2020Learning Joint 2D-3D Representations for Depth Completioncited by 71
- 2019Dense Depth Posterior (DDP) from Single Image and Sparse Rangecited by 57
- 2018Confidence Propagation through CNNs for Guided Sparse Depth Regressioncited by 52
- 2019An End-to-End Network for Panoptic Segmentationcited by 46
- 2021PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentationcited by 36
- 2021SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular imagescited by 33
- 2020Scaling Wide Residual Networks for Panoptic Segmentationcited by 32
- 2019Real-Time Panoptic Segmentation from Dense Detectionscited by 31
- 2022MGNet: Monocular Geometric Scene Understanding for Autonomous Drivingcited by 27
- 2019Fast Panoptic Segmentation Networkcited by 19
- 2022PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentationcited by 14
- 2019MultiDepth: Single-Image Depth Estimation via Multi-Task Regression and Classificationcited by 13
- 2022SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completioncited by 5
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.