mlsanityData to 7 Oct 2026

Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026

arXiv 2608.22337 ↗First posted 23 Aug 2026cs.MM cs.CV cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Speech-guided referring video object segmentation aims to recover the mask tracks of objects specified by a spoken motion description. Here, speech carries a linguistic instruction rather than acoustic evidence from a sounding object, so a solution must connect speech recognition, motion-centric temporal grounding, mask tracking, and explicit no-target handling. We introduce Speech2MaskTrack, our approach for the MeViS-Audio track of the 8th LSVOS Challenge. Speech2MaskTrack transcribes the spoken query and compiles it into structured constraints over category, count, direction, interaction role, and temporal phase. SAM3.1 enumerates multiple instance tracks, which TRACE ranks using complete-trajectory motion and relation evidence. A frozen lexical presence gate may suppress the ranked SAM3.1 base prediction. When the gate predicts that a target is present, an available full-expression-conditioned SaSaSa2VA track replaces the SAM3.1 mask. Only outputs that remain empty enter GPT-assisted recovery, which invokes SaSaSa2VA again under query- and mask-level verification. Speech2MaskTrack achieved second place in the official challenge ranking.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 34 references that are in the map. First posted on arXiv 23 Aug 2026.

Builds on

32 of 34 references.

References · 32

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  4. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  5. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  6. 2022Audio-Visual Segmentationcited by 148
  7. 2023Audio-Visual Segmentation with Semanticscited by 73
  8. 2021Positive Sample Propagation along the Audio-Visual Event Linecited by 69
  9. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  10. 2022Contrastive Positive Sample Propagation along the Audio-Visual Event Linecited by 50
  11. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  12. 2024TAVGBench: Benchmarking Text to Audible-Video Generationcited by 33
20 more
  1. 2024Towards Open-Vocabulary Audio-Visual Event Localizationcited by 26
  2. 2024Label-anticipated Event Disentanglement for Audio-Visual Video Parsingcited by 25
  3. 2025Multimodal Referring Segmentation: A Surveycited by 22
  4. 2023Audio-Visual Instance Segmentationcited by 21
  5. 2024Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labelingcited by 20
  6. 2023Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answeringcited by 19
  7. 2023Fine-grained Audible Video Descriptioncited by 17
  8. 2024Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsingcited by 13
  9. 2024Patch-level Sounding Object Tracking for Audio-Visual Question Answeringcited by 12
  10. 2024Dense Audio-Visual Event Localization under Cross-Modal Consistency and Multi-Temporal Granularity Collaborationcited by 12
  11. 2025Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptationcited by 10
  12. 2024LSVOS Challenge Report: Large-scale Complex and Long Video Object Segmentationcited by 8
  13. 2025The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VAcited by 7
  14. 2025Think Before You Segment: An Object-aware Reasoning Agent for Referring Audio-Visual Segmentationcited by 6
  15. 2025CLASP: Cross-modal Salient Anchor-based Semantic Propagation for Weakly-supervised Dense Audio-Visual Event Localizationcited by 6
  16. 2025SimToken: A Simple Baseline for Referring Audio-Visual Segmentationcited by 5
  17. 2025LSVOS 2025 Challenge Report: Recent Advances in Complex Video Object Segmentationcited by 4
  18. 2026MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generationcited by 3
  19. 2026Audit After Segmentation: Reference-Free Mask Quality Assessment for Language-Referred Audio-Visual Segmentationcited by 2
  20. 2026Face-Guided Sentiment Boundary Enhancement for Weakly-Supervised Temporal Sentiment Localizationcited by 1

Built on by

Not cited yet.