mlsanityData to 14 Oct 2026

MLVTG: Mamba-Based Feature Alignment and LLM-Driven Purification for Multi-Modal Video Temporal Grounding

arXiv 2506.08512 ↗First posted 10 Jun 2025cs.CV cs.AIcited by 0

On the map: State-space sequence models (S4, Mamba) → Video, motion and generative Mamba models

Abstract (arXiv)

Video Temporal Grounding (VTG), which aims to localize video clips corresponding to natural language queries, is a fundamental yet challenging task in video understanding. Existing Transformer-based methods often suffer from redundant attention and suboptimal multi-modal alignment. To address these limitations, we propose MLVTG, a novel framework that integrates two key modules: MambaAligner and LLMRefiner. MambaAligner uses stacked Vision Mamba blocks as a backbone instead of Transformers to model temporal dependencies and extract robust video representations for multi-modal alignment. LLMRefiner leverages the specific frozen layer of a pre-trained Large Language Model (LLM) to implicitly transfer semantic priors, enhancing multi-modal alignment without fine-tuning. This dual alignment strategy, temporal modeling via structured state-space dynamics and semantic purification via textual priors, enables more precise localization. Extensive experiments on QVHighlights, Charades-STA, and TVSum demonstrate that MLVTG achieves state-of-the-art performance and significantly outperforms existing baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 38 of its 47 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

38 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024VideoMamba: State Space Model for Efficient Video Understanding11 Mar

Other references · 37

  1. 2020End-to-End Object Detection with Transformerscited by 7,414
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  6. 2021ViViT: A Video Vision Transformercited by 1,620
  7. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  8. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  9. 2024VMamba: Visual State Space Modelcited by 969
  10. 2017Localizing Moments in Video with Natural Languagecited by 663
  11. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  12. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
25 more
  1. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  2. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  3. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  4. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  5. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  6. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
  7. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  8. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  9. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  10. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  11. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  12. 2022Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformercited by 101
  13. 2024VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Groundingcited by 85
  14. 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
  15. 2020MINI-Net: Multiple Instance Ranking Network for Video Highlight Detectioncited by 42
  16. 2016Video2GIF: Automatic Generation of Animated GIFs from Videocited by 41
  17. 2023MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformercited by 40
  18. 2023Frozen Transformers in Language Models Are Effective Visual Encoder Layerscited by 39
  19. 2021Cross-category Video Highlight Detection via Set-based Learningcited by 38
  20. 2020Learning Trailer Moments in Full-Length Moviescited by 37
  21. 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25
  22. 2024ViLLa: Video Reasoning Segmentation with Large Language Modelcited by 24
  23. 2022Learning Pixel-Level Distinctions for Video Highlight Detectioncited by 16
  24. 2022Probing Visual-Audio Representation for Video Highlight Detection via Hard-Pairs Guided Contrastive Learningcited by 9
  25. 2024MambaSCI: Efficient Mamba-UNet for Quad-Bayer Patterned Video Snapshot Compressive Imagingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MASRA: MLLM-Assisted Semantic-Relational Consistent Alignment for Video Temporal Grounding
  2. 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Models
  3. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Model
  4. 2024AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding
  5. 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
  6. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMs
  7. 2025HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling
  8. 2025Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding