# Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation

arXiv: https://arxiv.org/abs/2303.12782 · HTML: https://mlsanity.com/paper/2303.12782/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 77 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 30 of 70)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)

## Built on by (most cited first, 23 of 23)

- [MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions](https://mlsanity.com/paper/2308.08544/) (2023)
- [MatAnyone: Stable Video Matting with Consistent Memory Propagation](https://mlsanity.com/paper/2501.14677/) (2025)
- [Are They the Same? Exploring Visual Correspondence Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2501.04670/) (2025)
- [Exploiting Temporal State Space Sharing for Video Semantic Segmentation](https://mlsanity.com/paper/2503.20824/) (2025)
- [CAVIS: Context-Aware Video Instance Segmentation](https://mlsanity.com/paper/2407.03010/) (2024)
- [Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph Generation](https://mlsanity.com/paper/2412.07160/) (2024)
- [The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VA](https://mlsanity.com/paper/2509.16972/) (2025)
- [Integrating Boxes and Masks: A Multi-Object Framework for Unified Visual Tracking and Segmentation](https://mlsanity.com/paper/2308.13266/) (2023)
- [Iterative Robust Visual Grounding with Masked Reference based Centerpoint Supervision](https://mlsanity.com/paper/2307.12392/) (2023)
- [4th PVUW MeViS 3rd Place Report: Sa2VA](https://mlsanity.com/paper/2504.00476/) (2025)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [MED-VT++: Unifying Multimodal Learning with a Multiscale Encoder-Decoder Video Transformer](https://mlsanity.com/paper/2304.05930/) (2023)
- [Deep Spectral Improvement for Unsupervised Image Instance Segmentation](https://mlsanity.com/paper/2402.02474/) (2024)
- [PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning](https://mlsanity.com/paper/2407.17101/) (2024)
- [Semantic Segmentation on VSPW Dataset through Masked Video Consistency](https://mlsanity.com/paper/2406.04979/) (2024)
- [Rethinking Video Segmentation with Masked Video Consistency: Did the Model Learn as Intended?](https://mlsanity.com/paper/2408.10627/) (2024)
- [Improving Weakly-supervised Video Instance Segmentation by Leveraging Spatio-temporal Consistency](https://mlsanity.com/paper/2408.16661/) (2024)
- [Shift and matching queries for video semantic segmentation](https://mlsanity.com/paper/2410.07635/) (2024)
- [Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentation](https://mlsanity.com/paper/2410.12995/) (2024)
- [Local2Global query Alignment for Video Instance Segmentation](https://mlsanity.com/paper/2507.20120/) (2025)
- [Temporal Cluster Assignment for Efficient Real-Time Video Segmentation](https://mlsanity.com/paper/2508.05851/) (2025)
- [SaSaSaSa2VA: 2nd Place of the 5th PVUW MeViS-Text Track](https://mlsanity.com/paper/2603.27241/) (2026)
- [Hard-Region Supervision: #1 on the Waymo Open Dataset 2D Video Panoptic Segmentation Leaderboard](https://mlsanity.com/paper/2609.38714/) (2026)

## Abstract (arXiv)

Video segmentation aims to segment and track every pixel in diverse scenarios accurately. In this paper, we present Tube-Link, a versatile framework that addresses multiple core tasks of video segmentation with a unified architecture. Our framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks. To enhance the modeling of cross-tube relationships, we propose an effective way to perform tube-level linking via attention along the queries. In addition, we introduce temporal contrastive learning to instance-wise discriminative features for tube-level association. Our approach offers flexibility and efficiency for both short and long video inputs, as the length of each subclip can be varied according to the needs of datasets or scenarios. Tube-Link outperforms existing specialized architectures by a significant margin on five video segmentation datasets. Specifically, it achieves almost 13% relative improvements on VIPSeg and 4% improvements on KITTI-STEP over the strong baseline Video K-Net. When using a ResNet50 backbone on Youtube-VIS-2019 and 2021, Tube-Link boosts IDOL by 3% and 4%, respectively.