mlsanityData to 14 Oct 2026

Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs

arXiv 2506.22139 ↗First posted 27 Jun 2025cs.CVcited by 80

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated significant success in visual understanding tasks. However, challenges persist in adapting these models for video comprehension due to the large volume of data and temporal complexity. Existing Video-LLMs using uniform frame sampling often struggle to capture the query-related crucial spatiotemporal clues of videos effectively. In this paper, we introduce Q-Frame, a novel approach for adaptive frame selection and multi-resolution scaling tailored to the video's content and the specific query. Q-Frame employs a training-free, plug-and-play strategy generated by a text-image matching network like CLIP, utilizing the Gumbel-Max trick for efficient frame selection. Q-Frame allows Video-LLMs to process more frames without exceeding computational limits, thereby preserving critical temporal and spatial information. We demonstrate Q-Frame's effectiveness through extensive experiments on benchmark datasets, including MLVU, LongVideoBench, and Video-MME, illustrating its superiority over existing methods and its applicability across various video understanding tasks.

Key facts. Cited by 80 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 41 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

33 of 41 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  2. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input28 Aug
  3. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videos26 Aug
  4. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
  5. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  6. 2024Long Context Transfer from Language to Vision24 Jun
  7. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  8. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  9. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
  10. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  11. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  9. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  10. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  11. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  12. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
10 more
  1. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  2. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  3. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  4. 2023Video Understanding with Large Language Models: A Surveycited by 206
  5. 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 147
  6. 2018AdaFrame: Adaptive Frame Selection for Fast Video Recognitioncited by 134
  7. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
  8. 2019Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognitioncited by 66
  9. 2024KeyVideoLLM: Towards Large-scale Video Keyframe Selectioncited by 54
  10. 2021MGSampler: An Explainable Sampling Strategy for Video Action Recognitioncited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 62.

80 papers.

Sort:

By area:

  1. 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 37
  2. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 33
  3. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 29
  4. 2026Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representationcited by 21
  5. 2026LensWalk: Agentic Video Understanding by Planning How You See in Videoscited by 17
  6. 2025E-VRAG: Enhancing Long Video Understanding with Resource-Efficient Retrieval Augmented Generationcited by 12
  7. 2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understandingcited by 11
  8. 2025Video-EM: Event-Centric Episodic Memory for Long-Form Video Understandingcited by 10
  9. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understandingcited by 9
  10. 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understandingcited by 9
  11. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 9
  12. 2025Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-incited by 8
38 more
  1. 2026Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Searchcited by 8
  2. 2026GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understandingcited by 8
  3. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
  4. 2025The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VAcited by 7
  5. 2025FrameOracle: Learning What to See and How Much to See in Videoscited by 7
  6. 2026Event-Anchored Frame Selection for Effective Long-Video Understandingcited by 7
  7. 2026AdaCodec: A Predictive Visual Code for Video MLLMscited by 7
  8. 2026MSJoE: Jointly Evolving MLLM and Sampler for Efficient Long-Form Video Understandingcited by 6
  9. 2026Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Samplingcited by 5
  10. 2025LSVOS 2025 Challenge Report: Recent Advances in Complex Video Object Segmentationcited by 4
  11. 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 4
  12. 2026SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaborationcited by 4
  13. 2026Where to Focus: Query-Modulated Multimodal Keyframe Selection for Long Video Understandingcited by 4
  14. 2026OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Modelscited by 4
  15. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  16. 2025Enhancing Long Video Question Answering with Scene-Localized Frame Groupingcited by 3
  17. 2025A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answeringcited by 3
  18. 2026FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answeringcited by 3
  19. 2026Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understandingcited by 3
  20. 2026Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectorscited by 3
  21. 2026StreamScout: Learning When to Look Deeper for Streaming Video Understandingcited by 3
  22. 2025Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectoriescited by 2
  23. 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understandingcited by 2
  24. 2025Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrievalcited by 2
  25. 2026LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMscited by 2
  26. 2026Swift Sampling: Selecting Temporal Surprises via Taylor Seriescited by 2
  27. 2026PEEK: Picking Essential frames via Efficient Knowledge distillationcited by 2
  28. 2026MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answeringcited by 2
  29. 2026QCA: Query- and Content-Aware Keyframe Selection for Long Video Understandingcited by 2
  30. 2026ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QAcited by 2
  31. 2026LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videoscited by 2
  32. 2026LFS: Learnable Frame Selector for Event-Aware and Temporally Diverse Video Captioningcited by 1
  33. 2026Shot-Aware Frame Sampling for Video Understandingcited by 1
  34. 2026HORNet: Task-Guided Frame Selection for Video Question Answering with Vision-Language Modelscited by 1
  35. 2026Adaptive Greedy Frame Selection for Long Video Understandingcited by 1
  36. 2026CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answeringcited by 1
  37. 2026Decomposing Queries into Tool Calls for Long-Video Keyframe Retrievalcited by 1
  38. 2026MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMscited by 1

50 most cited of 80 All 80 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Think-Clip-Sample: Slow-Fast Frame Selection for Video Understanding
  2. 2025Adaptive Keyframe Sampling for Long Video Understanding
  3. 2025M-LLM Based Video Frame Selection for Efficient Video Understanding
  4. 2025Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning
  5. 2025BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding
  6. 2025K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding
  7. 2026QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understanding
  8. 2025Video-QTR: Query-Driven Temporal Reasoning Framework for Lightweight Video Understanding