mlsanityData to 14 Oct 2026

Generative Frame Sampler for Long Video Understanding

arXiv 2503.09146 ↗First posted 12 Mar 2025cs.CV cs.MMcited by 37

Not on the map. Why?

Abstract (arXiv)

Despite recent advances in Video Large Language Models (VideoLLMs), effectively understanding long-form videos remains a significant challenge. Perceiving lengthy videos containing thousands of frames poses substantial computational burden. To mitigate this issue, this paper introduces Generative Frame Sampler (GenS), a plug-and-play module integrated with VideoLLMs to facilitate efficient lengthy video perception. Built upon a lightweight VideoLLM, GenS leverages its inherent vision-language capabilities to identify question-relevant frames. To facilitate effective retrieval, we construct GenS-Video-150K, a large-scale video instruction dataset with dense frame relevance annotations. Extensive experiments demonstrate that GenS consistently boosts the performance of various VideoLLMs, including open-source models (Qwen2-VL-7B, Aria-25B, VILA-40B, LLaVA-Video-7B/72B) and proprietary assistants (GPT-4o, Gemini). When equipped with GenS, open-source VideoLLMs achieve impressive state-of-the-art results on long-form video benchmarks: LLaVA-Video-72B reaches 66.8 (+4.3) on LongVideoBench and 77.0 (+2.7) on MLVU, while Aria obtains 39.2 on HourVideo surpassing the Gemini-1.5-pro by 1.9 points. We will release all datasets and models at https://generative-sampler.github.io.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 37 of its 56 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

37 of 56 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  9. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  10. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  12. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
25 more
  1. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  2. 2017Localizing Moments in Video with Natural Languagecited by 663
  3. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  4. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  5. 2024Long Context Transfer from Language to Visioncited by 515
  6. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  7. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  8. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  9. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
  10. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  11. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  12. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  13. 2024Scaling Laws for Fine-Grained Mixture of Expertscited by 173
  14. 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 167
  15. 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 138
  16. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
  17. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  18. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 106
  19. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 102
  20. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 98
  21. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 70
  22. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  23. 2024Needle In A Multimodal Haystackcited by 58
  24. 2025MDP3: A Training-free Approach for List-wise Frame Selection in Video-LLMscited by 43
  25. 2024Temporal Reasoning Transfer from Text to Videocited by 38

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 21.

37 papers.

Sort:

By area:

  1. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 50
  2. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 38
  3. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 29
  4. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 29
  5. 2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 22
  6. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 16
  7. 2025See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Modelcited by 15
  8. 2025E-VRAG: Enhancing Long Video Understanding with Resource-Efficient Retrieval Augmented Generationcited by 12
  9. 2026TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captionscited by 11
  10. 2025Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenderscited by 10
  11. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understandingcited by 9
  12. 2026GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understandingcited by 8
25 more
  1. 2025KFS-Bench: Comprehensive Evaluation of Key Frame Sampling in Long Video Understandingcited by 6
  2. 2025TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understandingcited by 4
  3. 2025TempCore: Are Video QA Benchmarks Temporally Grounded?cited by 3
  4. 2025A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answeringcited by 3
  5. 2025Test-Time Temporal Sampling for Efficient MLLM Video Understandingcited by 3
  6. 2026FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answeringcited by 3
  7. 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understandingcited by 2
  8. 2026MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Callingcited by 2
  9. 2026AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understandingcited by 2
  10. 2026Swift Sampling: Selecting Temporal Surprises via Taylor Seriescited by 2
  11. 2026MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answeringcited by 2
  12. 2026Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memorycited by 1
  13. 2026Shot-Aware Frame Sampling for Video Understandingcited by 1
  14. 2026Adaptive Greedy Frame Selection for Long Video Understandingcited by 1
  15. 2026Rethinking RAG in Long Videos: What to Retrieve and How to Use It?cited by 1
  16. 2026VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understandingcited by 0
  17. 2026Zero-Shot 3D Question Answering via Hierarchical View-to-Token Transportationcited by 0
  18. 2026GOPAgen: Codec-Aware Agentic Long-Video Understanding with Structured Memorycited by 0
  19. 2026Gaussian Mixture Modeling for Event-Aware Visual Allocation in Long Video Understandingcited by 0
  20. 2026Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QAcited by 0
  21. 2026VisualRouter: Query-Grounded Visual Sampling for Long Video Understandingcited by 0
  22. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  23. 2026CueKFS: Agentic Cue-Driven Keyframe Selection for Long Video Understandingcited by 0
  24. 2026DynamicDx: Evaluating Evidence Acquisition in Video-Based Diagnosiscited by 0
  25. 2026Learning Temporal Evidence Selection for Video Anomaly Understandingcited by 0