# An End-to-End Trainable Video Panoptic Segmentation Method usingTransformers

arXiv: https://arxiv.org/abs/2110.04009 · HTML: https://mlsanity.com/paper/2110.04009/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 3 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 3 of 3)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [STEP: Segmenting and Tracking Every Pixel](https://mlsanity.com/paper/2102.11859/) (2021)

## Built on by (most cited first, 2 of 2)

- [Learning Appearance and Motion Cues for Panoptic Tracking](https://mlsanity.com/paper/2503.09191/) (2025)
- [Unified Perception: Efficient Depth-Aware Video Panoptic Segmentation with Minimal Annotation Costs](https://mlsanity.com/paper/2303.01991/) (2023)

## Abstract (arXiv)

In this paper, we present an algorithm to tackle a video panoptic segmentation problem, a newly emerging area of research. The video panoptic segmentation is a task that unifies the typical task of panoptic segmentation and multi-object tracking. In other words, it requires generating the instance tracking IDs along with panoptic segmentation results across video sequences. Our proposed video panoptic segmentation algorithm uses the transformer and it can be trained in end-to-end with an input of multiple video frames. We test our method on the STEP dataset and report its performance with recently proposed STQ metric. The method archived 57.81% on the KITTI-STEP dataset and 31.8% on the MOTChallenge-STEP dataset.