# 2nd Place Solution for MeViS Track in CVPR 2024 PVUW Workshop: Motion Expression guided Video Segmentation

arXiv: https://arxiv.org/abs/2406.13939 · HTML: https://mlsanity.com/paper/2406.13939/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 19 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2108.05565/) (2021)
- [MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions](https://mlsanity.com/paper/2308.08544/) (2023)
- [Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2003.08813/) (2020)
- [Universal Instance Perception as Object Discovery and Retrieval](https://mlsanity.com/paper/2303.06674/) (2023)
- [Language as Queries for Referring Video Object Segmentation](https://mlsanity.com/paper/2201.00487/) (2022)
- [Video Object Segmentation with Language Referring Expressions](https://mlsanity.com/paper/1803.08006/) (2018)
- [End-to-End Referring Video Object Segmentation with Multimodal Transformers](https://mlsanity.com/paper/2111.14821/) (2021)
- [Referring Image Segmentation via Cross-Modal Progressive Comprehension](https://mlsanity.com/paper/2010.00514/) (2020)
- [DVIS: Decoupled Video Instance Segmentation Framework](https://mlsanity.com/paper/2306.03413/) (2023)
- [General Object Foundation Model for Images and Videos at Scale](https://mlsanity.com/paper/2312.09158/) (2023)
- [OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation](https://mlsanity.com/paper/2307.09356/) (2023)
- [SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation](https://mlsanity.com/paper/2305.17011/) (2023)
- [Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation](https://mlsanity.com/paper/2305.16318/) (2023)
- [Spectrum-guided Multi-granularity Referring Video Object Segmentation](https://mlsanity.com/paper/2307.13537/) (2023)

## Built on by (most cited first, 2 of 2)

- [PVUW 2024 Challenge on Complex Video Understanding: Methods and Results](https://mlsanity.com/paper/2406.17005/) (2024)
- [The 2nd Solution for LSVOS Challenge RVOS Track: Spatial-temporal Refinement for Consistent Semantic Segmentation](https://mlsanity.com/paper/2408.12447/) (2024)

## Abstract (arXiv)

Motion Expression guided Video Segmentation is a challenging task that aims at segmenting objects in the video based on natural language expressions with motion descriptions. Unlike the previous referring video object segmentation (RVOS), this task focuses more on the motion in video content for language-guided video object segmentation, requiring an enhanced ability to model longer temporal, motion-oriented vision-language data. In this report, based on the RVOS methods, we successfully introduce mask information obtained from the video instance segmentation model as preliminary information for temporal enhancement and employ SAM for spatial refinement. Finally, our method achieved a score of 49.92 J &F in the validation phase and 54.20 J &F in the test phase, securing the final ranking of 2nd in the MeViS Track at the CVPR 2024 PVUW Challenge.