mlsanityData to 7 Oct 2026

Conditional Multi-Event Temporal Grounding in Long-Form Video

arXiv 2606.15320 ↗First posted 13 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositional temporal and spatial conditions. Existing benchmarks fall short: they localize only a single moment per query, count without temporal conditions, or treat grounding and counting as disjoint tasks. We introduce CoMET-Bench for Conditional Multi-Event Temporal Grounding in long-form video, comprising 2789 queries over 600 videos averaging 33.8 minutes across five real-world domains, with each query composed from 4 temporal conditions, 3 spatial conditions, and a dedicated negative-query subset. We further propose a unified evaluation protocol jointly measuring counting, grounding, and negative-query recognition, including a new Rejection-F1 metric that prevents trivial gaming by lazy "always-empty" models. Benchmarking a broad suite of MLLMs, agent-based, and grounding-specialized methods reveals that existing approaches remain far from solving this task. Building on these findings, we propose CoMET-Agent, a training-free agentic framework that reformulates the task as structured search-and-aggregate, improving [email protected] by 6.1% over GPT-5 purely through structural reasoning. Failure analysis further surfaces three open directions: fine-grained entity tracking, position-uniform retrieval, and causal event pairing.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 64 of its 71 references that are in the map. First posted on arXiv 13 Jun 2026.

Builds on

64 of 71 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms10 Apr
  2. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs16 Dec
  3. 2025Scaling RL to Long Videos10 Jul
  4. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streams30 Jun
  5. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Models21 Apr
  6. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding16 Dec
  7. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability27 Nov
  8. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
  9. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  12. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
2 more
  1. 2024LITA: Language Instructed Temporal-Localization Assistant27 Mar
  2. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec

Other references · 50

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  8. 2025DINOv3cited by 1,297
  9. 2025OpenAI GPT-5 System Cardcited by 910
  10. 2017Dense-Captioning Events in Videoscited by 789
  11. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  12. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
38 more
  1. 2017Localizing Moments in Video with Natural Languagecited by 660
  2. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  3. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  4. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  5. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  6. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  7. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  8. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
  9. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  10. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  11. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  12. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  13. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  14. 2024HourVideo: 1-Hour Video-Language Understandingcited by 110
  15. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 96
  16. 2024SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memorycited by 91
  17. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 87
  18. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
  19. 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
  20. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  21. 2024TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Modelscited by 71
  22. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
  23. 2020Counting Out Time: Class Agnostic Video Repetition Counting in the Wildcited by 67
  24. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  25. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  26. 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
  27. 2023AutoAD: Movie Description in Contextcited by 50
  28. 2022TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Countingcited by 42
  29. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 41
  30. 2020Context-aware and Scale-insensitive Temporal Repetition Countingcited by 33
  31. 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understandingcited by 30
  32. 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understandingcited by 25
  33. 2025MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrievalcited by 13
  34. 2026SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLMcited by 11
  35. 2024OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videoscited by 10
  36. 2025Moment of Untruth: Dealing with Negative Queries in Video Moment Retrievalcited by 9
  37. 2025DisTime: Distribution-based Time Representation for Video Large Language Modelscited by 7
  38. 2026Diagnosing Long-Video Quantitative Reasoning in Multimodal LLMs via Enumeration and Countingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards One-to-Many Temporal Grounding
  2. 2026DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding
  3. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMs
  4. 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Models
  5. 2026TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
  6. 2026Summarize Before Grounding: Query-Guided Chunk Condensation for Long-Video Temporal Grounding
  7. 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Models
  8. 2026OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding