mlsanityData to 7 Oct 2026

BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding

arXiv 2503.21483 ↗First posted 27 Mar 2025cs.CVcited by 72

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Large video-language models (VLMs) have demonstrated promising progress in various video understanding tasks. However, their effectiveness in long-form video analysis is constrained by limited context windows. Traditional approaches, such as uniform frame sampling, often inevitably allocate resources to irrelevant content, diminishing their effectiveness in real-world scenarios. In this paper, we introduce BOLT, a method to BOost Large VLMs without additional Training through a comprehensive study of frame selection strategies. First, to enable a more realistic evaluation of VLMs in long-form video understanding, we propose a multi-source retrieval evaluation setting. Our findings reveal that uniform sampling performs poorly in noisy contexts, underscoring the importance of selecting the right frames. Second, we explore several frame selection strategies based on query-frame similarity and analyze their effectiveness at inference time. Our results show that inverse transform sampling yields the most significant performance improvement, increasing accuracy on the Video-MME benchmark from 53.8% to 56.1% and MLVU benchmark from 58.9% to 63.4%. Our code is available at https://github.com/sming256/BOLT.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 40 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

35 of 40 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  2. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models9 Aug
  3. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  4. 2024Long Context Transfer from Language to Vision24 Jun
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  6. 2024MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens4 Apr
  7. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  8. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  10. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning27 Feb

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  6. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  8. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  9. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  10. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  11. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  12. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
13 more
  1. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  2. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  3. 2024SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Modelscited by 123
  4. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  5. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  6. 2024An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLMcited by 97
  7. 2023MM-VID: Advancing Video Understanding with GPT-4V(ision)cited by 82
  8. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 80
  9. 2024MoReVQA: Exploring Modular Reasoning Models for Video Question Answeringcited by 74
  10. 2023End-to-End Temporal Action Detection with 1B Parameters Across 1000 Framescited by 47
  11. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 43
  12. 2024Goldfish: Vision-Language Understanding of Arbitrarily Long Videoscited by 38
  13. 2024DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinementcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 23; 2026: 49.

72 papers.

Sort:

By area:

  1. 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 30
  2. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
  3. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 28
  4. 2025Temporal Chain of Thought: Long-Video Understanding by Thinking in Framescited by 27
  5. 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 17
  6. 2025ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understandingcited by 15
  7. 2026LensWalk: Agentic Video Understanding by Planning How You See in Videoscited by 14
  8. 2025E-VRAG: Enhancing Long Video Understanding with Resource-Efficient Retrieval Augmented Generationcited by 11
  9. 2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understandingcited by 11
  10. 2025Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenderscited by 10
  11. 2025Video-EM: Event-Centric Episodic Memory for Long-Form Video Understandingcited by 10
  12. 2026Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Searchcited by 8
38 more
  1. 2025FrameOracle: Learning What to See and How Much to See in Videoscited by 7
  2. 2026Event-Anchored Frame Selection for Effective Long-Video Understandingcited by 7
  3. 2026Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanismcited by 7
  4. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understandingcited by 6
  5. 2025KFS-Bench: Comprehensive Evaluation of Key Frame Sampling in Long Video Understandingcited by 6
  6. 2026MSJoE: Jointly Evolving MLLM and Sampler for Efficient Long-Form Video Understandingcited by 6
  7. 2025Frame Sampling Strategies Matter: A Benchmark for small vision language modelscited by 5
  8. 2025SeViCES: Unifying Semantic-Visual Evidence Consensus for Long Video Understandingcited by 5
  9. 2026CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Managementcited by 5
  10. 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMscited by 4
  11. 2025Venus: An Efficient Edge Memory-and-Retrieval System for VLM-based Online Video Understandingcited by 4
  12. 2025HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answeringcited by 4
  13. 2026Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Samplingcited by 4
  14. 2025Video-in-the-Loop: Span-Grounded Long Video QA with Interleaved Reasoningcited by 3
  15. 2025A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answeringcited by 3
  16. 2026QuPAINT: Physics-Aware Instruction Tuning Approach to Quantum Material Discoverycited by 3
  17. 2026HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answeringcited by 3
  18. 2026ViKey: Enhancing Temporal Understanding in Videos via Visual Promptingcited by 3
  19. 2026Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understandingcited by 3
  20. 2026Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectorscited by 3
  21. 2025Video Parallel Scaling: Aggregating Diverse Frame Subsets for VideoLLMscited by 2
  22. 2026FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understandingcited by 2
  23. 2026How to Train Your Long-Context Visual Document Modelcited by 2
  24. 2026A Multi-Agent Perception-Action Alliance for Efficient Long Video Reasoningcited by 2
  25. 2026ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QAcited by 2
  26. 2026LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videoscited by 2
  27. 2026StreamScout: Learning When to Look Deeper for Streaming Video Understandingcited by 2
  28. 2026MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Predictioncited by 1
  29. 2026HORNet: Task-Guided Frame Selection for Video Question Answering with Vision-Language Modelscited by 1
  30. 2026Adaptive Greedy Frame Selection for Long Video Understandingcited by 1
  31. 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Contextcited by 1
  32. 2026Swift Sampling: Selecting Temporal Surprises via Taylor Seriescited by 1
  33. 2026Decomposing Queries into Tool Calls for Long-Video Keyframe Retrievalcited by 1
  34. 2026MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answeringcited by 1
  35. 2026Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisitcited by 1
  36. 2026MMHBench: A Multi-Perspective Benchmark for Mental Health Understanding in Long-Form Videoscited by 1
  37. 2026When and Where to Look: Adaptive Visual Evidence Scheduling for Efficient Long Video Understandingcited by 1
  38. 2026One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understandingcited by 1

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Think-Clip-Sample: Slow-Fast Frame Selection for Video Understanding
  2. 2023Long Video Understanding with Learnable Retrieval in Video-Language Models
  3. 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understanding
  4. 2024Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models
  5. 2025Adaptive Keyframe Sampling for Long Video Understanding
  6. 2025From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignment
  7. 2025FiLA-Video: Spatio-Temporal Compression for Fine-Grained Long Video Understanding
  8. 2025RAG-Adapter: A Plug-and-Play RAG-enhanced Framework for Long Video Understanding