MLVTG: Mamba-Based Feature Alignment and LLM-Driven Purification for Multi-Modal Video Temporal Grounding
On the map: State-space sequence models (S4, Mamba) → Video, motion and generative Mamba models
Abstract (arXiv)
Video Temporal Grounding (VTG), which aims to localize video clips corresponding to natural language queries, is a fundamental yet challenging task in video understanding. Existing Transformer-based methods often suffer from redundant attention and suboptimal multi-modal alignment. To address these limitations, we propose MLVTG, a novel framework that integrates two key modules: MambaAligner and LLMRefiner. MambaAligner uses stacked Vision Mamba blocks as a backbone instead of Transformers to model temporal dependencies and extract robust video representations for multi-modal alignment. LLMRefiner leverages the specific frozen layer of a pre-trained Large Language Model (LLM) to implicitly transfer semantic priors, enhancing multi-modal alignment without fine-tuning. This dual alignment strategy, temporal modeling via structured state-space dynamics and semantic purification via textual priors, enables more precise localization. Extensive experiments on QVHighlights, Charades-STA, and TVSum demonstrate that MLVTG achieves state-of-the-art performance and significantly outperforms existing baselines.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 38 of its 47 references that are in the map. First posted on arXiv 10 Jun 2025.
Builds on
38 of 47 references.
Closest ancestors · 1
Same direction, newest first
Other references · 37
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2021ViViT: A Video Vision Transformercited by 1,620
- 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2024VMamba: Visual State Space Modelcited by 969
- 2017Localizing Moments in Video with Natural Languagecited by 663
- 2017TALL: Temporal Activity Localization via Language Querycited by 584
- 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
25 more
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
- 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
- 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
- 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
- 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
- 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
- 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
- 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
- 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
- 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
- 2022Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformercited by 101
- 2024VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Groundingcited by 85
- 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
- 2020MINI-Net: Multiple Instance Ranking Network for Video Highlight Detectioncited by 42
- 2016Video2GIF: Automatic Generation of Animated GIFs from Videocited by 41
- 2023MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformercited by 40
- 2023Frozen Transformers in Language Models Are Effective Visual Encoder Layerscited by 39
- 2021Cross-category Video Highlight Detection via Set-based Learningcited by 38
- 2020Learning Trailer Moments in Full-Length Moviescited by 37
- 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25
- 2024ViLLa: Video Reasoning Segmentation with Large Language Modelcited by 24
- 2022Learning Pixel-Level Distinctions for Video Highlight Detectioncited by 16
- 2022Probing Visual-Audio Representation for Video Highlight Detection via Hard-Pairs Guided Contrastive Learningcited by 9
- 2024MambaSCI: Efficient Mamba-UNet for Quad-Bayer Patterned Video Snapshot Compressive Imagingcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MASRA: MLLM-Assisted Semantic-Relational Consistent Alignment for Video Temporal Grounding
- 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Models
- 2025A Survey on Video Temporal Grounding with Multimodal Large Language Model
- 2024AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding
- 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
- 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMs
- 2025HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling
- 2025Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding