mlsanityData to 14 Oct 2026

LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos

arXiv 2607.25125 ↗First posted 27 Jul 2026cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Despite rapid progress in Multi-modal Large Language Models (MLLMs), understanding long-form videos is still bottlenecked by limited context windows. While recent keyframe sampling methods attempt to mitigate this by distilling video inputs into a compact set of query-relevant frames, navigating the vast spatio-temporal search space remains challenging, as spatial detail and temporal coverage often conflict. To address this, we introduce LENS, a training-free keyframe sampling framework that dynamically decides when to zoom in for fine-grained details and when to zoom out for broader context based on the text query. Concretely, LENS adaptively allocates a limited frame budget between spatial zoom-ins, which highlight query-relevant regions within individual frames, and temporal zoom-outs, which expand the temporal scope through multi-frame aggregation, enabling the model to reason across multiple granularities while capturing both high-fidelity details and long-range context. Across diverse long-form video benchmarks, LENS consistently outperforms prior state-of-the-art keyframe sampling methods and delivers substantial gains over uniform sampling, improving Video-MME accuracy from 53.3% to 60.7% with Qwen2.5-VL.Code is available at https://github.com/zhangce01/LENS.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 62 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

57 of 62 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs27 Jun
  2. 2025AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding16 Jun
  3. 2025Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation3 Apr
  4. 2025BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding27 Mar
  5. 2025Adaptive Keyframe Sampling for Long Video Understanding28 Feb
  6. 2025M-LLM Based Video Frame Selection for Efficient Video Understanding27 Feb
  7. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
  8. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  9. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  11. 2024Long Context Transfer from Language to Vision24 Jun
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
2 more
  1. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  2. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  10. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  11. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  12. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
31 more
  1. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  2. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  3. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  4. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  5. 2023Video Understanding with Large Language Models: A Surveycited by 206
  6. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
  7. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  8. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 178
  9. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
  10. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 117
  11. 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
  12. 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
  13. 2023End-to-End Temporal Action Detection with 1B Parameters Across 1000 Framescited by 47
  14. 2025Kwai Keye-VL Technical Reportcited by 47
  15. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  16. 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Modelscited by 41
  17. 2024Dual Prototype Evolving for Test-Time Generalization of Vision-Language Modelscited by 38
  18. 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
  19. 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understandingcited by 31
  20. 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
  21. 2024Attention Prompting on Image for Large Vision-Language Modelscited by 23
  22. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelitycited by 23
  23. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  24. 2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 22
  25. 2026PROGRESSLM: Towards Progress Reasoning in Vision-Language Modelscited by 15
  26. 2025∞-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidationcited by 14
  27. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forwardcited by 6
  28. 2025Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Ruminationcited by 6
  29. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  30. 2025When to Think and When to Look: Uncertainty-Guided Lookbackcited by 3
  31. 2026SPACENUM: Revisiting Spatial Numerical Understanding in VLMscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding
  2. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understanding
  3. 2025FOCUS: Efficient Keyframe Selection for Long Video Understanding
  4. 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understanding
  5. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning
  6. 2025Test-Time Temporal Sampling for Efficient MLLM Video Understanding
  7. 2025Iterative Zoom-In: Temporal Interval Exploration for Long Video Understanding
  8. 2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding