mlsanityData to 14 Oct 2026

Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understanding

arXiv 2503.13139 ↗First posted 17 Mar 2025cs.CV cs.AI cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Understanding long video content is a complex endeavor that often relies on densely sampled frame captions or end-to-end feature selectors, yet these techniques commonly overlook the logical relationships between textual queries and visual elements. In practice, computational constraints necessitate coarse frame subsampling, a challenge analogous to "finding a needle in a haystack." To address this issue, we introduce a semantics-driven search framework that reformulates keyframe selection under the paradigm of Visual Semantic-Logical Search. Specifically, we systematically define four fundamental logical dependencies: 1) spatial co-occurrence, 2) temporal proximity, 3) attribute dependency, and 4) causal order. These relations dynamically update frame sampling distributions through an iterative refinement process, enabling context-aware identification of semantically critical frames tailored to specific query requirements. Our method establishes new SOTA performance on the manually annotated benchmark in key-frame selection metrics. Furthermore, when applied to downstream video question-answering tasks, the proposed approach demonstrates the best performance gains over existing methods on LongVideoBench and Video-MME, validating its effectiveness in bridging the logical gap between textual queries and visual-temporal reasoning. The code will be publicly available.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 38 of its 53 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

38 of 53 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  7. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  8. 2023A Survey on Multimodal Large Language Modelscited by 909
  9. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  10. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  11. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  12. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
26 more
  1. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  2. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  3. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  4. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
  5. 2023Video Understanding with Large Language Models: A Surveycited by 206
  6. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
  7. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  8. 2024Streaming Long Video Understanding with Large Language Modelscited by 167
  9. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
  10. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 106
  11. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
  12. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  13. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  14. 2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Modelscited by 55
  15. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
  16. 2023Zero-Shot Video Question Answering with Procedural Programscited by 37
  17. 2024Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMscited by 37
  18. 2023GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generationcited by 36
  19. 2024Do You Remember? Dense Video Captioning with Cross-Modal Memory Retrievalcited by 36
  20. 2024OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquercited by 34
  21. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understandingcited by 30
  22. 2024VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMscited by 29
  23. 2023Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Modelscited by 27
  24. 2024Understanding Long Videos with Multimodal Language Modelscited by 25
  25. 2024End-to-End Video Question Answering with Frame Scoring Mechanisms and Adaptive Samplingcited by 19
  26. 2025A Survey of fMRI to Image Reconstructioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 21.

34 papers.

Sort:

By area:

  1. 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 37
  2. 2025GVPO: Group Variance Policy Optimization for Large Language Model Post-Trainingcited by 32
  3. 2025TSPO: Temporal Sampling Policy Optimization for Long-form Video Language Understandingcited by 25
  4. 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understandingcited by 25
  5. 2025See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Modelcited by 15
  6. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  7. 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understandingcited by 9
  8. 2025CyberV: Cybernetics for Test-time Scaling in Video Understandingcited by 8
  9. 2026GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understandingcited by 8
  10. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
  11. 2025APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrievalcited by 7
  12. 2025FrameOracle: Learning What to See and How Much to See in Videoscited by 7
22 more
  1. 2026Critic in the Loop: A Tri-System VLA Framework for Robust Long-Horizon Manipulationcited by 4
  2. 2026HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answeringcited by 4
  3. 2026Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Actioncited by 4
  4. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  5. 2026FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answeringcited by 3
  6. 2025Simple and Efficient Heterogeneous Temporal Graph Neural Networkcited by 2
  7. 2026PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Modelscited by 2
  8. 2026Ground, Cover, and Refine: Evidence-Centric Frame Selection for Long-Video Question Answeringcited by 2
  9. 2025VSI: Visual Subtitle Integration for Keyframe Selection to enhance Long Video Understandingcited by 1
  10. 2025SPIKE-RL: Video-LLMs meet Bayesian Surprisecited by 1
  11. 2025From Captions to Keyframes: KeyScore for Multimodal Frame Scoring and Video-Language Understandingcited by 1
  12. 2026LFS: Learnable Frame Selector for Event-Aware and Temporally Diverse Video Captioningcited by 1
  13. 2026Decomposing Queries into Tool Calls for Long-Video Keyframe Retrievalcited by 1
  14. 2026Active Inference for Micro-Gesture Recognition: EFE-Guided Temporal Sampling and Adaptive Learningcited by 0
  15. 2026GOPAgen: Codec-Aware Agentic Long-Video Understanding with Structured Memorycited by 0
  16. 2026QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understandingcited by 0
  17. 2026Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understandingcited by 0
  18. 2026Against Political Polarization: A Unified Framework for Tracing Evolving Political Ideologies on Social Mediacited by 0
  19. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  20. 2026CueKFS: Agentic Cue-Driven Keyframe Selection for Long Video Understandingcited by 0
  21. 2026Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learningcited by 0
  22. 2026Focus, Align, and Diffuse: Time-Series-Aware Keyframe Diffusion for Cardiac Dynamic Synthesis from Sparse Observationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Coverage-Driven Adaptive Keyframe Selection for Video Understanding
  2. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understanding
  3. 2025KFS-Bench: Comprehensive Evaluation of Key Frame Sampling in Long Video Understanding
  4. 2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding
  5. 2025K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding
  6. 2026Think-Clip-Sample: Slow-Fast Frame Selection for Video Understanding
  7. 2026Where to Focus: Query-Modulated Multimodal Keyframe Selection for Long Video Understanding
  8. 2025Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenders