# Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understanding

arXiv: https://arxiv.org/abs/2503.13139 · HTML: https://mlsanity.com/paper/2503.13139/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 53 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models](https://mlsanity.com/paper/2311.17043/) (2023)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos](https://mlsanity.com/paper/2405.19209/) (2024)
- [VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding](https://mlsanity.com/paper/2403.11481/) (2024)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents](https://mlsanity.com/paper/2311.05437/) (2023)
- [MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition](https://mlsanity.com/paper/2201.08383/) (2022)
- [Streaming Long Video Understanding with Large Language Models](https://mlsanity.com/paper/2405.16009/) (2024)
- [LongVLM: Efficient Long Video Understanding via Large Language Models](https://mlsanity.com/paper/2404.03384/) (2024)
- [TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning](https://mlsanity.com/paper/2410.19702/) (2024)
- [Frame-Voyager: Learning to Query Frames for Video Large Language Models](https://mlsanity.com/paper/2410.03226/) (2024)
- [Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA](https://mlsanity.com/paper/2406.09396/) (2024)
- [Koala: Key frame-conditioned long video-LLM](https://mlsanity.com/paper/2404.04346/) (2024)
- [Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2406.11230/) (2024)
- [ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos](https://mlsanity.com/paper/2406.19392/) (2024)
- [Zero-Shot Video Question Answering with Procedural Programs](https://mlsanity.com/paper/2312.00937/) (2023)
- [Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs](https://mlsanity.com/paper/2406.09367/) (2024)
- [GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation](https://mlsanity.com/paper/2311.16511/) (2023)

## Built on by (most cited first, 30 of 34)

- [FOCUS: Efficient Keyframe Selection for Long Video Understanding](https://mlsanity.com/paper/2510.27280/) (2025)
- [GVPO: Group Variance Policy Optimization for Large Language Model Post-Training](https://mlsanity.com/paper/2504.19599/) (2025)
- [TSPO: Temporal Sampling Policy Optimization for Long-form Video Language Understanding](https://mlsanity.com/paper/2508.04369/) (2025)
- [VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding](https://mlsanity.com/paper/2512.12360/) (2025)
- [See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model](https://mlsanity.com/paper/2509.16087/) (2025)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understanding](https://mlsanity.com/paper/2510.02778/) (2025)
- [CyberV: Cybernetics for Test-time Scaling in Video Understanding](https://mlsanity.com/paper/2506.07971/) (2025)
- [GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding](https://mlsanity.com/paper/2603.25072/) (2026)
- [ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning](https://mlsanity.com/paper/2603.28610/) (2026)
- [APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval](https://mlsanity.com/paper/2506.04953/) (2025)
- [FrameOracle: Learning What to See and How Much to See in Videos](https://mlsanity.com/paper/2510.03584/) (2025)
- [Critic in the Loop: A Tri-System VLA Framework for Robust Long-Horizon Manipulation](https://mlsanity.com/paper/2603.05185/) (2026)
- [HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering](https://mlsanity.com/paper/2603.18558/) (2026)
- [Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Action](https://mlsanity.com/paper/2605.22283/) (2026)
- [Watch, Remember, Reason: Human-View Video Understanding with MLLMs](https://mlsanity.com/paper/2606.07433/) (2026)
- [FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answering](https://mlsanity.com/paper/2603.04349/) (2026)
- [Simple and Efficient Heterogeneous Temporal Graph Neural Network](https://mlsanity.com/paper/2510.18467/) (2025)
- [PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models](https://mlsanity.com/paper/2606.03598/) (2026)
- [Ground, Cover, and Refine: Evidence-Centric Frame Selection for Long-Video Question Answering](https://mlsanity.com/paper/2608.01660/) (2026)
- [VSI: Visual Subtitle Integration for Keyframe Selection to enhance Long Video Understanding](https://mlsanity.com/paper/2508.06869/) (2025)
- [SPIKE-RL: Video-LLMs meet Bayesian Surprise](https://mlsanity.com/paper/2509.23433/) (2025)
- [From Captions to Keyframes: KeyScore for Multimodal Frame Scoring and Video-Language Understanding](https://mlsanity.com/paper/2510.06509/) (2025)
- [LFS: Learnable Frame Selector for Event-Aware and Temporally Diverse Video Captioning](https://mlsanity.com/paper/2601.14594/) (2026)
- [Decomposing Queries into Tool Calls for Long-Video Keyframe Retrieval](https://mlsanity.com/paper/2605.23826/) (2026)
- [Active Inference for Micro-Gesture Recognition: EFE-Guided Temporal Sampling and Adaptive Learning](https://mlsanity.com/paper/2603.07559/) (2026)
- [GOPAgen: Codec-Aware Agentic Long-Video Understanding with Structured Memory](https://mlsanity.com/paper/2606.06532/) (2026)
- [QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understanding](https://mlsanity.com/paper/2607.04559/) (2026)
- [Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding](https://mlsanity.com/paper/2607.24794/) (2026)
- [Against Political Polarization: A Unified Framework for Tracing Evolving Political Ideologies on Social Media](https://mlsanity.com/paper/2608.17987/) (2026)

## Abstract (arXiv)

Understanding long video content is a complex endeavor that often relies on densely sampled frame captions or end-to-end feature selectors, yet these techniques commonly overlook the logical relationships between textual queries and visual elements. In practice, computational constraints necessitate coarse frame subsampling, a challenge analogous to "finding a needle in a haystack." To address this issue, we introduce a semantics-driven search framework that reformulates keyframe selection under the paradigm of Visual Semantic-Logical Search. Specifically, we systematically define four fundamental logical dependencies: 1) spatial co-occurrence, 2) temporal proximity, 3) attribute dependency, and 4) causal order. These relations dynamically update frame sampling distributions through an iterative refinement process, enabling context-aware identification of semantically critical frames tailored to specific query requirements. Our method establishes new SOTA performance on the manually annotated benchmark in key-frame selection metrics. Furthermore, when applied to downstream video question-answering tasks, the proposed approach demonstrates the best performance gains over existing methods on LongVideoBench and Video-MME, validating its effectiveness in bridging the logical gap between textual queries and visual-temporal reasoning. The code will be publicly available.