# Robust Promptable Video Object Segmentation

arXiv: https://arxiv.org/abs/2605.12006 · HTML: https://mlsanity.com/paper/2605.12006/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 51 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 30 of 39)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [Putting the Object Back into Video Object Segmentation](https://mlsanity.com/paper/2310.12982/) (2023)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree](https://mlsanity.com/paper/2410.16268/) (2024)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning](https://mlsanity.com/paper/2312.11420/) (2023)
- [Learning Motion Patterns in Videos](https://mlsanity.com/paper/1612.07217/) (2016)
- [RobustSAM: Segment Anything Robustly on Degraded Images](https://mlsanity.com/paper/2406.09627/) (2024)
- [FIFO: Learning Fog-invariant Features for Foggy Scene Segmentation](https://mlsanity.com/paper/2204.01587/) (2022)
- [URIE: Universal Image Enhancement for Visual Recognition in the Wild](https://mlsanity.com/paper/2007.08979/) (2020)
- [Human Pose Estimation in Extremely Low-Light Conditions](https://mlsanity.com/paper/2303.15410/) (2023)
- [UniVS: Unified and Universal Video Segmentation with Prompts as Queries](https://mlsanity.com/paper/2402.18115/) (2024)
- [Event-guided Deblurring of Unknown Exposure Time Videos](https://mlsanity.com/paper/2112.06988/) (2021)
- [Event-assisted Low-Light Video Object Segmentation](https://mlsanity.com/paper/2404.01945/) (2024)
- [ACDC: The Adverse Conditions Dataset with Correspondences for Robust Semantic Driving Scene Perception](https://mlsanity.com/paper/2104.13395/) (2021)
- [Efficient Track Anything](https://mlsanity.com/paper/2411.18933/) (2024)
- [Parameter-Efficient Tuning on Layer Normalization for Pre-trained Language Models](https://mlsanity.com/paper/2211.08682/) (2022)
- [LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration](https://mlsanity.com/paper/2410.15385/) (2024)
- [On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers](https://mlsanity.com/paper/2308.09610/) (2023)
- [Vanishing-Point-Guided Video Semantic Segmentation of Driving Scenes](https://mlsanity.com/paper/2401.15261/) (2024)
- [ReferEverything: Towards Segmenting Everything We Can Speak of in Videos](https://mlsanity.com/paper/2410.23287/) (2024)

## Abstract (arXiv)

The performance of promptable video object segmentation (PVOS) models substantially degrades under input corruptions, which prevents PVOS deployment in safety-critical domains. This paper offers the first comprehensive study on robust PVOS (RobustPVOS). We first construct a new, comprehensive benchmark with two real-world evaluation datasets of 351 video clips and more than 2,500 object masks under real-world adverse conditions. At the same time, we generate synthetic training data by applying diverse and temporally varying corruptions to existing VOS datasets. Moreover, we present a new RobustPVOS method, dubbed Memory-object-conditioned Gated-rank Adaptation (MoGA). The key to successfully performing RobustPVOS is two-fold: effectively handling object-specific degradation and ensuring temporal consistency in predictions. MoGA leverages object-specific representations maintained in memory across frames to condition the robustification process, which allows the model to handle each tracked object differently in a temporally consistent way. Extensive experiments on our benchmark validate MoGA's efficacy, showing consistent and significant improvements across diverse corruption types on both synthetic and real-world datasets, establishing a strong baseline for future RobustPVOS research. Our benchmark is publicly available at https://sohyun-l.github.io/RobustPVOS_project_page/.