mlsanityData to 14 Oct 2026

MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions

arXiv 2308.08544 ↗First posted 16 Aug 2023cs.CVcited by 228

Not on the map. Why?

Cited by 41 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper strives for motion expressions guided video segmentation, which focuses on segmenting objects in video content based on a sentence describing the motion of the objects. Existing referring video object datasets typically focus on salient objects and use language expressions that contain excessive static attributes that could potentially enable the target object to be identified in a single frame. These datasets downplay the importance of motion in video content for language-guided video object segmentation. To investigate the feasibility of using motion expressions to ground and segment objects in videos, we propose a large-scale dataset called MeViS, which contains numerous motion expressions to indicate target objects in complex environments. We benchmarked 5 existing referring video object segmentation (RVOS) methods and conducted a comprehensive comparison on the MeViS dataset. The results show that current RVOS methods cannot effectively address motion expression-guided video segmentation. We further analyze the challenges and propose a baseline approach for the proposed MeViS dataset. The goal of our benchmark is to provide a platform that enables the development of effective language-guided video segmentation algorithms that leverage motion expressions as a primary cue for object segmentation in complex video scenes. The proposed MeViS dataset has been released at https://henghuiding.github.io/MeViS.

Key facts. Cited by 228 papers in the map (data through 14 Oct 2026). Builds on 48 of its 65 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

48 of 65 references.

References · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2017Mask R-CNNcited by 7,277
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  8. 2016Modeling Context in Referring Expressionscited by 1,063
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  10. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  11. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  12. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
36 more
  1. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  2. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  3. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  4. 2016Segmentation from Natural Language Expressionscited by 257
  5. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  6. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  7. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  8. 2022Language as Queries for Referring Video Object Segmentationcited by 182
  9. 2018Video Object Segmentation with Language Referring Expressionscited by 162
  10. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  11. 2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
  12. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  13. 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
  14. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  15. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  16. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  17. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  18. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 111
  19. 2019Boundary-Aware Feature Propagation for Scene Segmentationcited by 107
  20. 2018Actor and Action Video Segmentation from a Sentencecited by 106
  21. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  22. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  23. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  24. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
  25. 2022VITA: Video Instance Segmentation via Object Token Associationcited by 94
  26. 2020Human-centric Spatio-Temporal Video Grounding With Visual Transformerscited by 85
  27. 2018Dynamic Multimodal Instance Segmentation guided by natural language queriescited by 83
  28. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  29. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  30. 2021Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentationcited by 42
  31. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  32. 2021Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentationcited by 32
  33. 2022Multi-Attention Network for Compressed Video Referring Object Segmentationcited by 28
  34. 2023Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentationcited by 23
  35. 2022Modeling Motion with Multi-Modal Features for Text-Based Video Segmentationcited by 18
  36. 2020Reducing the Annotation Effort for Video Object Segmentation Datasetscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 55; 2025: 97; 2026: 67.

228 papers.

Sort:

By area:

  1. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  2. 2023Video Understanding with Large Language Models: A Surveycited by 206
  3. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
  4. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  5. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  6. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 127
  7. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  8. 2024Efficient Multimodal Large Language Models: A Surveycited by 96
  9. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 75
  10. 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 72
  11. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
  12. 2024SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentationcited by 64
38 more
  1. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
  2. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 61
  3. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  4. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  5. 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
  6. 2024Toward Visual Grounding: A Surveycited by 52
  7. 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
  8. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 51
  9. 2023Merlin:Empowering Multimodal LLMs with Foresight Mindscited by 46
  10. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 44
  11. 2023GREC: Generalized Referring Expression Comprehensioncited by 42
  12. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
  13. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  14. 2024SegPoint: Segment Any Point Cloud via Large Language Modelcited by 35
  15. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 35
  16. 2023Ref-Diff: Zero-shot Referring Image Segmentation with Generative Modelscited by 33
  17. 2024Ref-AVS: Refer and Segment Objects in Audio-Visual Scenescited by 33
  18. 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Modelcited by 33
  19. 2025Unhackable Temporal Rewarding for Scalable Video MLLMscited by 33
  20. 2025ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundationscited by 30
  21. 2023SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Processcited by 29
  22. 2024Effectiveness Assessment of Recent Large Vision-Language Modelscited by 29
  23. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29
  24. 2025The Devil is in Temporal Token: High Quality Video Reasoning Segmentationcited by 26
  25. 2025IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasiscited by 26
  26. 2024Artemis: Towards Referential Understanding in Complex Videoscited by 25
  27. 2024RefMask3D: Language-Guided Transformer for 3D Referring Segmentationcited by 25
  28. 2025TransAnimate: Taming Layer Diffusion to Generate RGBA Videocited by 24
  29. 2024Bootstrapping Referring Multi-Object Trackingcited by 22
  30. 2025Multimodal Referring Segmentation: A Surveycited by 22
  31. 2025PixelRefer: A Unified Framework for Spatio-Temporal Object Referring with Arbitrary Granularitycited by 21
  32. 2023QDFormer: Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decompositioncited by 20
  33. 2024SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flowcited by 20
  34. 2025Toward Generalizable Forgery Detection and Reasoningcited by 20
  35. 2025ReferSplat: Referring Segmentation in 3D Gaussian Splattingcited by 20
  36. 2023VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Modelscited by 19
  37. 2024EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Drivingcited by 19
  38. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19

50 most cited of 228 All 228 (JSON)