# RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation

arXiv: https://arxiv.org/abs/2305.01255 · HTML: https://mlsanity.com/paper/2305.01255/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [ParseNet: Looking Wider to See Better](https://mlsanity.com/paper/1506.04579/) (2015)
- [Seamless Scene Segmentation](https://mlsanity.com/paper/1905.01220/) (2019)
- [EfficientPS: Efficient Panoptic Segmentation](https://mlsanity.com/paper/2004.02307/) (2020)
- [Fully Convolutional Networks for Panoptic Segmentation](https://mlsanity.com/paper/2012.00720/) (2020)
- [SSAP: Single-Shot Instance Segmentation With Affinity Pyramid](https://mlsanity.com/paper/1909.01616/) (2019)
- [DeeperLab: Single-Shot Image Parser](https://mlsanity.com/paper/1902.05093/) (2019)
- [CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation](https://mlsanity.com/paper/2206.08948/) (2022)
- [Scaling Wide Residual Networks for Panoptic Segmentation](https://mlsanity.com/paper/2011.11675/) (2020)
- [Real-Time Panoptic Segmentation from Dense Detections](https://mlsanity.com/paper/1912.01202/) (2019)
- [RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer](https://mlsanity.com/paper/2210.07124/) (2022)
- [MGNet: Monocular Geometric Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2206.13199/) (2022)
- [Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images](https://mlsanity.com/paper/2304.08222/) (2023)

## Built on by (most cited first, 1 of 1)

- [MGNiceNet: Unified Monocular Geometric Scene Understanding](https://mlsanity.com/paper/2411.11466/) (2024)

## Abstract (arXiv)

Panoptic segmentation is one of the most challenging scene parsing tasks, combining the tasks of semantic segmentation and instance segmentation. While much progress has been made, few works focus on the real-time application of panoptic segmentation methods. In this paper, we revisit the recently introduced K-Net architecture. We propose vital changes to the architecture, training, and inference procedure, which massively decrease latency and improve performance. Our resulting RT-K-Net sets a new state-of-the-art performance for real-time panoptic segmentation methods on the Cityscapes dataset and shows promising results on the challenging Mapillary Vistas dataset. On Cityscapes, RT-K-Net reaches 60.2 % PQ with an average inference time of 32 ms for full resolution 1024x2048 pixel images on a single Titan RTX GPU. On Mapillary Vistas, RT-K-Net reaches 33.2 % PQ with an average inference time of 69 ms. Source code is available at https://github.com/markusschoen/RT-K-Net.