# Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network

arXiv: https://arxiv.org/abs/1809.02110 · HTML: https://mlsanity.com/paper/1809.02110/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [Pointly-Supervised Panoptic Segmentation](https://mlsanity.com/paper/2210.13950/) (2022)
- [SANeRF-HQ: Segment Anything for NeRF in High Quality](https://mlsanity.com/paper/2312.01531/) (2023)
- [PanSt3R: Multi-view Consistent Panoptic Segmentation](https://mlsanity.com/paper/2506.21348/) (2025)
- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [Towards Real-Time Panoptic Narrative Grounding by an End-to-End Grounding Network](https://mlsanity.com/paper/2301.03160/) (2023)
- [Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images](https://mlsanity.com/paper/2304.08222/) (2023)
- [SE-PSNet: Silhouette-based Enhancement Feature for Panoptic Segmentation Network](https://mlsanity.com/paper/2107.05093/) (2021)
- [TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency](https://mlsanity.com/paper/2210.05152/) (2022)

## Abstract (arXiv)

We present a single network method for panoptic segmentation. This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics. Joint training is the first step towards an end-to-end panoptic segmentation network and is faster and more memory efficient than training and predicting with two networks, as done in previous work. The architecture consists of a ResNet-50 feature extractor shared by the semantic segmentation and instance segmentation branch. For instance segmentation, a Mask R-CNN type of architecture is used, while the semantic segmentation branch is augmented with a Pyramid Pooling Module. Results for this method are submitted to the COCO and Mapillary Joint Recognition Challenge 2018. Our approach achieves a PQ score of 17.6 on the Mapillary Vistas validation set and 27.2 on the COCO test-dev set.