# SUNet: Scale-aware Unified Network for Panoptic Segmentation

arXiv: https://arxiv.org/abs/2209.02877 · HTML: https://mlsanity.com/paper/2209.02877/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 44 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Seamless Scene Segmentation](https://mlsanity.com/paper/1905.01220/) (2019)
- [EfficientPS: Efficient Panoptic Segmentation](https://mlsanity.com/paper/2004.02307/) (2020)
- [Fully Convolutional Networks for Panoptic Segmentation](https://mlsanity.com/paper/2012.00720/) (2020)
- [Attention-guided Unified Network for Panoptic Segmentation](https://mlsanity.com/paper/1812.03904/) (2018)
- [AdaptIS: Adaptive Instance Selection Network](https://mlsanity.com/paper/1909.07829/) (2019)
- [An End-to-End Network for Panoptic Segmentation](https://mlsanity.com/paper/1903.05027/) (2019)
- [Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras](https://mlsanity.com/paper/1801.00708/) (2018)
- [Unifying Training and Inference for Panoptic Segmentation](https://mlsanity.com/paper/2001.04982/) (2020)
- [Real-Time Panoptic Segmentation from Dense Detections](https://mlsanity.com/paper/1912.01202/) (2019)
- [Learning Instance Occlusion for Panoptic Segmentation](https://mlsanity.com/paper/1906.05896/) (2019)

## Abstract (arXiv)

Panoptic segmentation combines the advantages of semantic and instance segmentation, which can provide both pixel-level and instance-level environmental perception information for intelligent vehicles. However, it is challenged with segmenting objects of various scales, especially on extremely large and small ones. In this work, we propose two lightweight modules to mitigate this problem. First, Pixel-relation Block is designed to model global context information for large-scale things, which is based on a query-independent formulation and brings small parameter increments. Then, Convectional Network is constructed to collect extra high-resolution information for small-scale stuff, supplying more appropriate semantic features for the downstream segmentation branches. Based on these two modules, we present an end-to-end Scale-aware Unified Network (SUNet), which is more adaptable to multi-scale objects. Extensive experiments on Cityscapes and COCO demonstrate the effectiveness of the proposed methods.