# Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images

arXiv: https://arxiv.org/abs/2304.08222 · HTML: https://mlsanity.com/paper/2304.08222/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Seamless Scene Segmentation](https://mlsanity.com/paper/1905.01220/) (2019)
- [EfficientPS: Efficient Panoptic Segmentation](https://mlsanity.com/paper/2004.02307/) (2020)
- [Fully Convolutional Networks for Panoptic Segmentation](https://mlsanity.com/paper/2012.00720/) (2020)
- [SSAP: Single-Shot Instance Segmentation With Affinity Pyramid](https://mlsanity.com/paper/1909.01616/) (2019)
- [Attention-guided Unified Network for Panoptic Segmentation](https://mlsanity.com/paper/1812.03904/) (2018)
- [AdaptIS: Adaptive Instance Selection Network](https://mlsanity.com/paper/1909.07829/) (2019)
- [DeeperLab: Single-Shot Image Parser](https://mlsanity.com/paper/1902.05093/) (2019)
- [An End-to-End Network for Panoptic Segmentation](https://mlsanity.com/paper/1903.05027/) (2019)
- [Unifying Training and Inference for Panoptic Segmentation](https://mlsanity.com/paper/2001.04982/) (2020)
- [Learning to Fuse Things and Stuff](https://mlsanity.com/paper/1812.01192/) (2018)
- [SOGNet: Scene Overlap Graph Network for Panoptic Segmentation](https://mlsanity.com/paper/1911.07527/) (2019)
- [Fast Panoptic Segmentation Network](https://mlsanity.com/paper/1910.03892/) (2019)
- [Improving Panoptic Segmentation at All Scales](https://mlsanity.com/paper/2012.07717/) (2020)
- [Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network](https://mlsanity.com/paper/1809.02110/) (2018)

## Built on by (most cited first, 3 of 3)

- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Rethinking Interactive Image Segmentation with Low Latency, High Quality, and Diverse Prompts](https://mlsanity.com/paper/2404.00741/) (2024)
- [RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation](https://mlsanity.com/paper/2305.01255/) (2023)

## Abstract (arXiv)

Unified panoptic segmentation methods are achieving state-of-the-art results on several datasets. To achieve these results on high-resolution datasets, these methods apply crop-based training. In this work, we find that, although crop-based training is advantageous in general, it also has a harmful side-effect. Specifically, it limits the ability of unified networks to discriminate between large object instances, causing them to make predictions that are confused between multiple instances. To solve this, we propose Intra-Batch Supervision (IBS), which improves a network's ability to discriminate between instances by introducing additional supervision using multiple images from the same batch. We show that, with our IBS, we successfully address the confusion problem and consistently improve the performance of unified networks. For the high-resolution Cityscapes and Mapillary Vistas datasets, we achieve improvements of up to +2.5 on the Panoptic Quality for thing classes, and even more considerable gains of up to +5.8 on both the pixel accuracy and pixel precision, which we identify as better metrics to capture the confusion problem.