LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
Despite rapid progress in Multi-modal Large Language Models (MLLMs), understanding long-form videos is still bottlenecked by limited context windows. While recent keyframe sampling methods attempt to mitigate this by distilling video inputs into a compact set of query-relevant frames, navigating the vast spatio-temporal search space remains challenging, as spatial detail and temporal coverage often conflict. To address this, we introduce LENS, a training-free keyframe sampling framework that dynamically decides when to zoom in for fine-grained details and when to zoom out for broader context based on the text query. Concretely, LENS adaptively allocates a limited frame budget between spatial zoom-ins, which highlight query-relevant regions within individual frames, and temporal zoom-outs, which expand the temporal scope through multi-frame aggregation, enabling the model to reason across multiple granularities while capturing both high-fidelity details and long-range context. Across diverse long-form video benchmarks, LENS consistently outperforms prior state-of-the-art keyframe sampling methods and delivers substantial gains over uniform sampling, improving Video-MME accuracy from 53.3% to 60.7% with Qwen2.5-VL.Code is available at https://github.com/zhangce01/LENS.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 62 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
57 of 62 references.
Closest ancestors · 14
Same direction, newest first
- 2025Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs27 Jun
- 2025AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding16 Jun
- 2025Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation3 Apr
- 2025BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding27 Mar
- 2025Adaptive Keyframe Sampling for Long Video Understanding28 Feb
- 2025M-LLM Based Video Frame Selection for Efficient Video Understanding27 Feb
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
- 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
- 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024Long Context Transfer from Language to Vision24 Jun
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
2 more
Other references · 43
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2023Instruction Tuning for Large Language Models: A Surveycited by 623
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
31 more
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
- 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
- 2023Video Understanding with Large Language Models: A Surveycited by 206
- 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
- 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 178
- 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
- 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 117
- 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
- 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
- 2023End-to-End Temporal Action Detection with 1B Parameters Across 1000 Framescited by 47
- 2025Kwai Keye-VL Technical Reportcited by 47
- 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
- 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Modelscited by 41
- 2024Dual Prototype Evolving for Test-Time Generalization of Vision-Language Modelscited by 38
- 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
- 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understandingcited by 31
- 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
- 2024Attention Prompting on Image for Large Vision-Language Modelscited by 23
- 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelitycited by 23
- 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
- 2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 22
- 2026PROGRESSLM: Towards Progress Reasoning in Vision-Language Modelscited by 15
- 2025∞-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidationcited by 14
- 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forwardcited by 6
- 2025Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Ruminationcited by 6
- 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
- 2025When to Think and When to Look: Uncertainty-Guided Lookbackcited by 3
- 2026SPACENUM: Revisiting Spatial Numerical Understanding in VLMscited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding
- 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understanding
- 2025FOCUS: Efficient Keyframe Selection for Long Video Understanding
- 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understanding
- 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning
- 2025Test-Time Temporal Sampling for Efficient MLLM Video Understanding
- 2025Iterative Zoom-In: Temporal Interval Exploration for Long Video Understanding
- 2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding