mlsanityData to 7 Oct 2026

Efficient Unified Multimodal Understanding (EUMU): Winning Solution for the MUMU Track at the 8th LSVOS Challenge

arXiv 2609.19451 ↗First posted 16 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The Mobile Unified Multimodal Understanding (MUMU) Challenge requires a single efficient model to jointly perform multi-concept image tagging, open-vocabulary object detection, and image captioning. We present Efficient Unified Multimodal Understanding (EUMU), the winning solution for the MUMU Track of the 8th LSVOS Challenge. EUMU builds on a shared pretrained multimodal model, using its prompt-based capabilities for detection and captioning and training lightweight heads on shared visual features to predict quality, scene, and event tags. Rather than treating the three tasks independently, EUMU applies task-aware inference refinement by reusing task outputs as cross-task cues. For detection, caption cues help recover objects missed by the initial detection. For captioning, detection cues help refine the caption to better reflect the detected objects. For tagging, image statistics refine quality predictions, while caption and detection cues refine scene and event predictions. This design unifies all three tasks within a single model while satisfying the challenge's resource constraints. EUMU contains 239.169M parameters, requires 23.947 GFLOPs, uses 4.5 GB of peak inference memory, and achieves a final challenge score of 17.3409. Code and models are available at https://github.com/Dayoung-Kil/EUMU.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

12 of 16 references.

References · 12

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  3. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  4. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  5. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  6. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  7. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  8. 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 351
  9. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  10. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  11. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  12. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34

Built on by

Not cited yet.