Towards One-to-Many Temporal Grounding
Not on the map. Why?
Abstract (arXiv)
Temporal Grounding (TG) aims to localize video segments corresponding to a textual query. Prior research predominantly focuses on single-segment retrieval. Real-world scenarios, however, often require localizing multiple disjoint segments for a single query -- a setting we term One-to-Many Temporal Grounding (OMTG). Previous state-of-the-art MLLMs, optimized for one-to-one settings, struggle in this context, often yielding near-zero scores due to a lack of event cardinality perception. To bridge this gap, we present a systematic solution with three key contributions. First, we establish the first comprehensive OMTG benchmark, introducing Count Accuracy (C-Acc) and Effective Temporal F1 (EtF1) as evaluation metrics. Second, we curate a high-quality OMTG dataset comprising 56k samples through a sophisticated construction pipeline. Third, we develop novel temporal and caption reward functions specifically designed for OMTG. In particular, the caption reward leverages Chain-of-Thought reasoning over dense video captions to explicitly guide policy optimization toward both preciseness and completeness. Extensive experiments show our model achieves a new state-of-the-art EtF1 of 43.65% on OMTG Bench, outperforming Gemini 2.5 Pro and Seed-1.8 by 15.85% and 15.61%, respectively. Project Page: https://insomniaaac.github.io/OMTG/
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
42 of 54 references.
References · 42
- 2023GPT-4 Technical Reportcited by 18,396
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023VideoChat: Chat-Centric Video Understandingcited by 857
- 2017Dense-Captioning Events in Videoscited by 793
30 more
- 2017TALL: Temporal Activity Localization via Language Querycited by 584
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 255
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
- 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
- 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
- 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
- 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 165
- 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 146
- 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
- 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
- 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 106
- 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 89
- 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
- 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learningcited by 69
- 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 65
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 50
- 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
- 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidencecited by 49
- 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Modelscited by 28
- 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 27
- 2024LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrievalcited by 17
- 2024Question-Answering Dense Video Eventscited by 11
- 2025CyberV: Cybernetics for Test-time Scaling in Video Understandingcited by 8
- 2024Vid-Morp: Video Moment Retrieval Pretraining from Unlabeled Videos in the Wildcited by 5
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026TempCloze: Can Video-LLMs Identify the Missing Middle?cited by 0
- 2026Grounding with Confidence: Controllable Generative Video Temporal Groundingcited by 0
- 2026Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Groundingcited by 0
- 2026Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidencecited by 0
- 2026Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence13 Oct 2026
- 2026Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Grounding6 Oct 2026
- 2026Grounding with Confidence: Controllable Generative Video Temporal Grounding30 Sep 2026
- 2026TempCloze: Can Video-LLMs Identify the Missing Middle?1 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding
- 2026Conditional Multi-Event Temporal Grounding in Long-Form Video
- 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMs
- 2025TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding
- 2025Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding
- 2025OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios
- 2026DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding
- 2025TimeScope: Towards Task-Oriented Temporal Grounding In Long Videos