# An End-to-End Network for Panoptic Segmentation

arXiv: https://arxiv.org/abs/1903.05027 · HTML: https://mlsanity.com/paper/1903.05027/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2019.

## Built on by (most cited first, 30 of 46)

- [Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP](https://mlsanity.com/paper/2308.02487/) (2023)
- [Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation](https://mlsanity.com/paper/2103.14962/) (2021)
- [A Generalist Framework for Panoptic Segmentation of Images and Videos](https://mlsanity.com/paper/2210.06366/) (2022)
- [Open-World Entity Segmentation](https://mlsanity.com/paper/2107.14228/) (2021)
- [CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation](https://mlsanity.com/paper/2206.08948/) (2022)
- [Waymo Open Dataset: Panoramic Video Panoptic Segmentation](https://mlsanity.com/paper/2206.07704/) (2022)
- [Part-aware Panoptic Segmentation](https://mlsanity.com/paper/2106.06351/) (2021)
- [Cross-View Regularization for Domain Adaptive Panoptic Segmentation](https://mlsanity.com/paper/2103.02584/) (2021)
- [Exemplar-Based Open-Set Panoptic Segmentation Network](https://mlsanity.com/paper/2105.08336/) (2021)
- [PATS: Patch Area Transportation with Subdivision for Local Feature Matching](https://mlsanity.com/paper/2303.07700/) (2023)
- [CLUSTSEG: Clustering for Universal Segmentation](https://mlsanity.com/paper/2305.02187/) (2023)
- [Instance-wise Occlusion and Depth Orders in Natural Scenes](https://mlsanity.com/paper/2111.14562/) (2021)
- [MGNet: Monocular Geometric Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2206.13199/) (2022)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [PanSt3R: Multi-view Consistent Panoptic Segmentation](https://mlsanity.com/paper/2506.21348/) (2025)
- [DFormer: Diffusion-guided Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2306.03437/) (2023)
- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentation](https://mlsanity.com/paper/2306.17319/) (2023)
- [PUPS: Point Cloud Unified Panoptic Segmentation](https://mlsanity.com/paper/2302.06185/) (2023)
- [Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches](https://mlsanity.com/paper/2206.08016/) (2022)
- [PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentation](https://mlsanity.com/paper/2206.00468/) (2022)
- [Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning](https://mlsanity.com/paper/2206.10711/) (2022)
- [Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center](https://mlsanity.com/paper/2406.10527/) (2024)
- [kMaX-DeepLab: k-means Mask Transformer](https://mlsanity.com/paper/2207.04044/) (2022)
- [LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network](https://mlsanity.com/paper/2203.07186/) (2022)
- [Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentation](https://mlsanity.com/paper/2204.05370/) (2022)
- [NAS-FCOS: Efficient Search for Object Detection Architectures](https://mlsanity.com/paper/2110.12423/) (2021)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [Multi-task Fusion for Efficient Panoptic-Part Segmentation](https://mlsanity.com/paper/2212.07671/) (2022)
- [Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer](https://mlsanity.com/paper/2101.10620/) (2021)

## Abstract (arXiv)

Panoptic segmentation, which needs to assign a category label to each pixel and segment each object instance simultaneously, is a challenging topic. Traditionally, the existing approaches utilize two independent models without sharing features, which makes the pipeline inefficient to implement. In addition, a heuristic method is usually employed to merge the results. However, the overlapping relationship between object instances is difficult to determine without sufficient context information during the merging process. To address the problems, we propose a novel end-to-end network for panoptic segmentation, which can efficiently and effectively predict both the instance and stuff segmentation in a single network. Moreover, we introduce a novel spatial ranking module to deal with the occlusion problem between the predicted instances. Extensive experiments have been done to validate the performance of our proposed method and promising results have been achieved on the COCO Panoptic benchmark.