Data to 7 Oct 2026

T*: Re-thinking Temporal Search for Long-Form Video Understanding

arXiv 2504.02259 ↗First posted 3 Apr 2025cs.CVcited by 82

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Efficiently understanding long-form videos remains a significant challenge in computer vision. In this work, we revisit temporal search paradigms for long-form video understanding and address a fundamental issue pertaining to all state-of-the-art (SOTA) long-context vision-language models (VLMs). Our contributions are twofold: First, we frame temporal search as a Long Video Haystack problem: finding a minimal set of relevant frames (e.g., one to five) from tens of thousands based on specific queries. Upon this formulation, we introduce LV-Haystack, the first dataset with 480 hours of videos, 15,092 human-annotated instances for both training and evaluation aiming to improve temporal search quality and efficiency. Results on LV-Haystack highlight a significant research gap in temporal search capabilities, with current SOTA search methods only achieving 2.1% temporal F1 score on the Longvideobench subset. Next, inspired by visual search in images, we propose a lightweight temporal search framework, T* that reframes costly temporal search as spatial search. T* leverages powerful visual localization techniques commonly used in images and introduces an adaptive zooming-in mechanism that operates across both temporal and spatial dimensions. Extensive experiments show that integrating T* with existing methods significantly improves SOTA long-form video understanding. Under an inference budget of 32 frames, T* improves GPT-4o's performance from 50.5% to 53.1% and LLaVA-OneVision-OV-72B's performance from 56.5% to 62.4% on the Longvideobench XL subset. Our code, benchmark, and models are provided in the Supplementary material.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 93 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

38 of 93 references.

Closest ancestors · 1

Same direction, newest first

2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos29 May

Other references · 37

2025Qwen2.5-VL Technical Reportcited by 5,418
2020Longformer: The Long-Document Transformercited by 3,362
2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
2023VideoChat: Chat-Centric Video Understandingcited by 851
2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402

25 more

2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 112
2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 80
2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
2023Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capabilitycited by 62
2024Needle In A Multimodal Haystackcited by 56
2024Language Repository for Long Video Understandingcited by 55
2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Modelscited by 53
2024Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMscited by 35
2024Needle in the Haystack for Memory Based Large Language Modelscited by 32
2024Improving Gloss-free Sign Language Translation by Reducing Representation Densitycited by 30
2024Understanding Long Videos with Multimodal Language Modelscited by 25
2023Discovering Spatio-Temporal Rationales for Video Question Answeringcited by 21
2024End-to-End Video Question Answering with Frame Scoring Mechanisms and Adaptive Samplingcited by 19
2023Cross-modality Data Augmentation for End-to-End Sign Language Translationcited by 16
2024VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understandingcited by 10
2023Long Video Understanding with Learnable Retrieval in Video-Language Modelscited by 9
2024Question-Instructed Visual Descriptions for Zero-Shot Video Question Answeringcited by 8
2024LLMs Meet Long Video: Advancing Long Video Question Answering with An Interactive Visual Adapter in LLMscited by 5
2023Towards Weakly Supervised End-to-end Learning for Long-video Action Recognitioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 52.

82 papers.

Sort:

By area:

2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 67
2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
2025ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understandingcited by 50
2025VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoningcited by 32
2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 26
2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 26
2025TSPO: Temporal Sampling Policy Optimization for Long-form Video Language Understandingcited by 23
2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 21
2025TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learningcited by 18
2025ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understandingcited by 15
2025Vidi: Large Multimodal Models for Video Understanding and Editingcited by 14
2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understandingcited by 11

38 more

50 most cited of 82 All 82 (JSON)

Similar, not cited

Close in wording, no citation either way