mlsanityData to 7 Oct 2026

2nd of the 5th PVUW MeViS-Audio Track: ASR-SaSaSa2VA

arXiv 2604.23935 ↗First posted 27 Apr 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Audio-based video object segmentation aims to locate and segment objects in videos conditioned on audio cues, requiring precise understanding of both appearance and motion. Recent audio-driven video segmentation methods extend MLLMs by fusing audio and visual features for end-to-end localization. Despite their promise, these approaches are computationally intensive, struggle with aligning temporal audio cues to dynamic video content, and depend on large paired audio-video datasets. To address these challenges, we present ASR-SaSaSa2VA, a resource-efficient framework for audio-guided video segmentation. The key idea is to convert audio inputs into textual motion descriptions via automatic speech recognition (ASR) models and then leverage pre-trained text-based referring video segmentation models (e.g., SaSaSa2VA) for pixel-level predictions. To further enhance robustness, we incorporate a no-target expression detection module, implemented by a fine-tuned audio-based MLLM, which filters out audio clips that do not refer to any target object. This design allows the system to exploit strong pre-trained models while effectively handling ambiguous or irrelevant audio inputs. Our approach achieves a final score of 80.7 in the 5th PVUW Challenge (MeViS-v2-Audio track), earning the second-place ranking.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 14 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

12 of 14 references.

References · 12

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  2. 2023A Survey on Multimodal Large Language Modelscited by 898
  3. 2020A Survey on Visual Transformercited by 653
  4. 2025Qwen3-Omni Technical Reportcited by 451
  5. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  6. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
  7. 2026Qwen3-ASR Technical Reportcited by 105
  8. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  9. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  10. 2023Audio-Visual Instance Segmentationcited by 21
  11. 2022Dynamic Contrastive Distillation for Image-Text Retrievalcited by 14
  12. 2025The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VAcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026MLLM-Assisted Audio VOS: A 3rd Place Report for the MeViS-Audio Track, 8th LSVOS Challenge
  2. 2025SimToken: A Simple Baseline for Referring Audio-Visual Segmentation
  3. 2025From Waveforms to Pixels: A Survey on Audio-Visual Segmentation
  4. 2023BAVS: Bootstrapping Audio-Visual Segmentation by Integrating Foundation Knowledge
  5. 2026Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation
  6. 2025The 1st Solution for 4th PVUW MeViS Challenge: Unleashing the Potential of Large Multimodal Models for Referring Video Segmentation
  7. 2024Open-Vocabulary Audio-Visual Semantic Segmentation
  8. 2024Can Textual Semantics Mitigate Sounding Object Segmentation Preference?