# Joint Inductive and Transductive Learning for Video Object Segmentation

arXiv: https://arxiv.org/abs/2108.03679 · HTML: https://mlsanity.com/paper/2108.03679/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 62 references that are in the map.
- First posted on arXiv 8 Aug 2021.

## Builds on (most cited first, 30 of 51)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Space-Time Correspondence as a Contrastive Random Walk](https://mlsanity.com/paper/2006.14613/) (2020)
- [One-Shot Video Object Segmentation](https://mlsanity.com/paper/1611.05198/) (2016)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [Siam R-CNN: Visual Tracking by Re-Detection](https://mlsanity.com/paper/1911.12836/) (2019)
- [Tracking Emerges by Colorizing Videos](https://mlsanity.com/paper/1806.09594/) (2018)
- [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016)
- [Collaborative Video Object Segmentation by Foreground-Background Integration](https://mlsanity.com/paper/2003.08333/) (2020)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [Online Adaptation of Convolutional Neural Networks for Video Object Segmentation](https://mlsanity.com/paper/1706.09364/) (2017)

## Built on by (most cited first, 30 of 61)

- [MovieChat: From Dense Token to Sparse Memory for Long Video Understanding](https://mlsanity.com/paper/2307.16449/) (2023)
- [XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model](https://mlsanity.com/paper/2207.07115/) (2022)
- [Putting the Object Back into Video Object Segmentation](https://mlsanity.com/paper/2310.12982/) (2023)
- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [LVOS: A Benchmark for Long-term Video Object Segmentation](https://mlsanity.com/paper/2211.10181/) (2022)
- [XMem++: Production-level Video Segmentation From Few Annotated Frames](https://mlsanity.com/paper/2307.15958/) (2023)
- [Recurrent Dynamic Embedding for Video Object Segmentation](https://mlsanity.com/paper/2205.03761/) (2022)
- [LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation](https://mlsanity.com/paper/2404.19326/) (2024)
- [Per-Clip Video Object Segmentation](https://mlsanity.com/paper/2208.01924/) (2022)
- [Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2207.08485/) (2022)
- [Reliable Propagation-Correction Modulation for Video Object Segmentation](https://mlsanity.com/paper/2112.02853/) (2021)
- [Learning Quality-aware Dynamic Memory for Video Object Segmentation](https://mlsanity.com/paper/2207.07922/) (2022)
- [Look Before You Match: Instance Understanding Matters in Video Object Segmentation](https://mlsanity.com/paper/2212.06826/) (2022)
- [Unsupervised Video Object Segmentation via Prototype Memory Network](https://mlsanity.com/paper/2209.03712/) (2022)
- [Scalable Video Object Segmentation with Simplified Framework](https://mlsanity.com/paper/2308.09903/) (2023)
- [SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximization](https://mlsanity.com/paper/2208.10128/) (2022)
- [Global Spectral Filter Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2210.05567/) (2022)
- [UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces](https://mlsanity.com/paper/2312.15715/) (2023)
- [Event-assisted Low-Light Video Object Segmentation](https://mlsanity.com/paper/2404.01945/) (2024)
- [Boosting Video Object Segmentation via Space-time Correspondence Learning](https://mlsanity.com/paper/2304.06211/) (2023)
- [Region Aware Video Object Segmentation with Deep Motion Modeling](https://mlsanity.com/paper/2207.10258/) (2022)
- [Scalable Video Object Segmentation with Identification Mechanism](https://mlsanity.com/paper/2203.11442/) (2022)
- [PanoVOS: Bridging Non-panoramic and Panoramic Views with Transformer for Video Segmentation](https://mlsanity.com/paper/2309.12303/) (2023)
- [Tackling Background Distraction in Video Object Segmentation](https://mlsanity.com/paper/2207.06953/) (2022)
- [MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation](https://mlsanity.com/paper/2303.07815/) (2023)
- [Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentation](https://mlsanity.com/paper/2303.10100/) (2023)
- [Detect Any Shadow: Segment Anything for Video Shadow Detection](https://mlsanity.com/paper/2305.16698/) (2023)
- [BATMAN: Bilateral Attention Transformer in Motion-Appearance Neighboring Space for Video Object Segmentation](https://mlsanity.com/paper/2208.01159/) (2022)
- [Towards Robust Video Object Segmentation with Adaptive Object Calibration](https://mlsanity.com/paper/2207.00887/) (2022)
- [360VOTS: Visual Object Tracking and Segmentation in Omnidirectional Videos](https://mlsanity.com/paper/2404.13953/) (2024)

## Abstract (arXiv)

Semi-supervised video object segmentation is a task of segmenting the target object in a video sequence given only a mask annotation in the first frame. The limited information available makes it an extremely challenging task. Most previous best-performing methods adopt matching-based transductive reasoning or online inductive learning. Nevertheless, they are either less discriminative for similar instances or insufficient in the utilization of spatio-temporal information. In this work, we propose to integrate transductive and inductive learning into a unified framework to exploit the complementarity between them for accurate and robust video object segmentation. The proposed approach consists of two functional branches. The transduction branch adopts a lightweight transformer architecture to aggregate rich spatio-temporal cues while the induction branch performs online inductive learning to obtain discriminative target information. To bridge these two diverse branches, a two-head label encoder is introduced to learn the suitable target prior for each of them. The generated mask encodings are further forced to be disentangled to better retain their complementarity. Extensive experiments on several prevalent benchmarks show that, without the need of synthetic training data, the proposed approach sets a series of new state-of-the-art records. Code is available at https://github.com/maoyunyao/JOINT.