# LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval

arXiv: https://arxiv.org/abs/2411.14505 · HTML: https://mlsanity.com/paper/2411.14505/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 21 Nov 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)

## Built on by (most cited first, 17 of 17)

- [Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark](https://mlsanity.com/paper/2504.14693/) (2025)
- [A Survey on Video Temporal Grounding with Multimodal Large Language Model](https://mlsanity.com/paper/2508.10922/) (2025)
- [Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models](https://mlsanity.com/paper/2511.23478/) (2025)
- [DisTime: Distribution-based Time Representation for Video Large Language Models](https://mlsanity.com/paper/2505.24329/) (2025)
- [Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding](https://mlsanity.com/paper/2510.20244/) (2025)
- [Towards One-to-Many Temporal Grounding](https://mlsanity.com/paper/2606.06294/) (2026)
- [Watch, Remember, Reason: Human-View Video Understanding with MLLMs](https://mlsanity.com/paper/2606.07433/) (2026)
- [Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMs](https://mlsanity.com/paper/2503.09027/) (2025)
- [Sparse-Dense Side-Tuner for efficient Video Temporal Grounding](https://mlsanity.com/paper/2507.07744/) (2025)
- [DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs](https://mlsanity.com/paper/2607.12503/) (2026)
- [Augmenting Moment Retrieval: Zero-Dependency Two-Stage Learning](https://mlsanity.com/paper/2510.19622/) (2025)
- [SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM](https://mlsanity.com/paper/2511.14143/) (2025)
- [See More, Store Less: Memory-Efficient Resolution for Video Moment Retrieval](https://mlsanity.com/paper/2601.09350/) (2026)
- [Moment and Highlight Detection via MLLM Frame Segmentation](https://mlsanity.com/paper/2512.12246/) (2025)
- [SiMing-Bench: Evaluating Procedural Correctness from Continuous Interactions in Clinical Skill Videos](https://mlsanity.com/paper/2604.09037/) (2026)
- [EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization](https://mlsanity.com/paper/2607.04872/) (2026)
- [VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding](https://mlsanity.com/paper/2610.01766/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) are widely used for visual perception, understanding, and reasoning. However, long video processing and precise moment retrieval remain challenging due to LLMs' limited context size and coarse frame extraction. We propose the Large Language-and-Vision Assistant for Moment Retrieval (LLaVA-MR), which enables accurate moment retrieval and contextual grounding in videos using MLLMs. LLaVA-MR combines Dense Frame and Time Encoding (DFTE) for spatial-temporal feature extraction, Informative Frame Selection (IFS) for capturing brief visual and motion patterns, and Dynamic Token Compression (DTC) to manage LLM context limitations. Evaluations on benchmarks like Charades-STA and QVHighlights demonstrate that LLaVA-MR outperforms 11 state-of-the-art methods, achieving an improvement of 1.82% in R1@0.5 and 1.29% in mAP@0.5 on the QVHighlights dataset. Our implementation will be open-sourced upon acceptance.