mlsanityData to 7 Oct 2026

TRACE: Temporal Grounding Video LLM via Causal Event Modeling

arXiv 2410.05643 ↗First posted 8 Oct 2024cs.CVcited by 87

Not on the map. Why?

Cited by 25 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Video Temporal Grounding (VTG) is a crucial capability for video understanding models and plays a vital role in downstream tasks such as video browsing and editing. To effectively handle various tasks simultaneously and enable zero-shot prediction, there is a growing trend in employing video LLMs for VTG tasks. However, current video LLM-based methods rely exclusively on natural language generation, lacking the ability to model the clear structure inherent in videos, which restricts their effectiveness in tackling VTG tasks. To address this issue, this paper first formally introduces causal event modeling framework, which represents video LLM outputs as sequences of events, and predict the current event using previous events, video inputs, and textural instructions. Each event consists of three components: timestamps, salient scores, and textual captions. We then propose a novel task-interleaved video LLM called TRACE to effectively implement the causal event modeling framework in practice. The TRACE process visual frames, timestamps, salient scores, and text as distinct tasks, employing various encoders and decoding heads for each. Task tokens are arranged in an interleaved sequence according to the causal event modeling framework's formulation. Extensive experiments on various VTG tasks and datasets demonstrate the superior performance of TRACE compared to state-of-the-art video LLMs. Our model and code are available at https://github.com/gyxxyg/TRACE.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 66 of its 81 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

66 of 81 references.

References · 66

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2023Mistral 7Bcited by 3,065
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  10. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  11. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  12. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
38 more
  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  3. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  4. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  5. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  6. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  7. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  8. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  9. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  10. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  11. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  12. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  13. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  14. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  15. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  16. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  17. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  18. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  19. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  20. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
  21. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  22. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  23. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  24. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  25. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  26. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
  27. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  28. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  29. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Modelcited by 145
  30. 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
  31. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
  32. 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
  33. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  34. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  35. 2024VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Groundingcited by 85
  36. 2018Localizing Moments in Video with Temporal Languagecited by 83
  37. 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
  38. 2024HawkEye: Training Video-Text LLMs for Grounding Text in Videoscited by 83

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 38; 2026: 45.

87 papers.

Sort:

By area:

  1. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  2. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 66
  3. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  4. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 57
  5. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  6. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 49
  7. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
  8. 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidencecited by 43
  9. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 41
  10. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  11. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 31
  12. 2025SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capabilitycited by 28
38 more
  1. 2025VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generationcited by 22
  2. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 19
  3. 2024TimeRefine: Temporal Grounding with Time Refining Video LLMcited by 15
  4. 2025TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMscited by 15
  5. 2025VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Treecited by 15
  6. 2024Do Language Models Understand Time?cited by 14
  7. 2025TimeExpert: An Expert-Guided Video LLM for Video Temporal Groundingcited by 14
  8. 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
  9. 2025TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Actioncited by 13
  10. 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Groundingcited by 11
  11. 2025iMOVE: Instance-Motion-Aware Video Understandingcited by 10
  12. 2025Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Promptingcited by 9
  13. 2026VISD: Enhancing Video Reasoning via Structured Self-Distillationcited by 9
  14. 2025Watch and Listen: Understanding Audio-Visual-Speech Moments with Multimodal LLMcited by 7
  15. 2025VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotationscited by 7
  16. 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learningcited by 7
  17. 2025ToG-Bench: Task-Oriented Spatio-Temporal Grounding in Egocentric Videoscited by 6
  18. 2026CutClaw: Agentic Hours-Long Video Editing via Music Synchronizationcited by 6
  19. 2025TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Groundingcited by 5
  20. 2025TimeScope: Towards Task-Oriented Temporal Grounding In Long Videoscited by 5
  21. 2025ChronusOmni: Improving Time Awareness of Omni Large Language Modelscited by 5
  22. 2026VideoLoom: A Video Large Language Model for Joint Spatial-Temporal Understandingcited by 5
  23. 2026GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinkingcited by 5
  24. 2025SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situationcited by 4
  25. 2025LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answeringcited by 4
  26. 2025OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarioscited by 4
  27. 2025Factorized Learning for Temporally Grounded Video-Language Modelscited by 4
  28. 2026CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Modelscited by 4
  29. 2026ArrowGEV: Grounding Events in Video via Learning the Arrow of Timecited by 4
  30. 2026OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Groundingcited by 4
  31. 2025A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answeringcited by 3
  32. 2026STVG-R1: Incentivizing Instance-Level Reasoning and Grounding in Videos via Reinforcement Learningcited by 3
  33. 2026VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Promptingcited by 3
  34. 2026Bridging Time and Space: Decoupled Spatio-Temporal Alignment for Video Groundingcited by 3
  35. 2026EvoGround: Self-Evolving Video Agents for Video Temporal Groundingcited by 3
  36. 2026Temporal-Aware Reasoning Optimization for Video Temporal Groundingcited by 3
  37. 2025Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMscited by 2
  38. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2

50 most cited of 87 All 87 (JSON)