mlsanityData to 14 Oct 2026

Knowing Where to Focus: Event-aware Transformer for Video Grounding

arXiv 2308.06947 ↗First posted 14 Aug 2023cs.CV cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

Recent DETR-based video grounding models have made the model directly predict moment timestamps without any hand-crafted components, such as a pre-defined proposal or non-maximum suppression, by learning moment queries. However, their input-agnostic moment queries inevitably overlook an intrinsic temporal structure of a video, providing limited positional information. In this paper, we formulate an event-aware dynamic moment query to enable the model to take the input-specific content and positional information of the video into account. To this end, we present two levels of reasoning: 1) Event reasoning that captures distinctive event units constituting a given video using a slot attention mechanism; and 2) moment reasoning that fuses the moment queries with a given sentence through a gated fusion transformer layer and learns interactions between the moment queries and video-sentence representations to predict moment timestamps. Extensive experiments demonstrate the effectiveness and efficiency of the event-aware dynamic moment queries, outperforming state-of-the-art approaches on several video grounding benchmarks.

Key facts. Cited by 79 papers in the map (data through 14 Oct 2026). Builds on 71 of its 91 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

71 of 91 references.

References · 71

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2018SlowFast Networks for Video Recognitioncited by 1,867
  7. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  8. 2020Object-Centric Learning with Slot Attentioncited by 876
  9. 2017Dense-Captioning Events in Videoscited by 793
  10. 2017Localizing Moments in Video with Natural Languagecited by 663
  11. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  12. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
38 more
  1. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  2. 2021Conditional DETR for Fast Training Convergencecited by 424
  3. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  4. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  5. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  6. 2020Local-Global Video-Text Interactions for Temporal Groundingcited by 188
  7. 2020Dense Regression Network for Video Groundingcited by 176
  8. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  9. 2020Support-set bottlenecks for video-text representation learningcited by 170
  10. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  11. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  12. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  13. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  14. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  15. 2020Learning the Best Pooling Strategy for Visual Semantic Embeddingcited by 137
  16. 2019Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videoscited by 135
  17. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  18. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  19. 2019CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrievalcited by 110
  20. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
  21. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
  22. 2018Timeception for Complex Action Recognitioncited by 89
  23. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  24. 2018MAC: Mining Activity Concepts for Language-based Temporal Localizationcited by 84
  25. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  26. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  27. 2019Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attentioncited by 81
  28. 2019Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videoscited by 79
  29. 2020Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localizationcited by 74
  30. 2021QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queriescited by 72
  31. 2020Counting Out Time: Class Agnostic Video Repetition Counting in the Wildcited by 68
  32. 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 65
  33. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  34. 2020Uncovering Hidden Challenges in Query-Based Video Moment Retrievalcited by 58
  35. 2022Accelerating DETR Convergence via Semantic-Aligned Matchingcited by 58
  36. 2021On Pursuit of Designing Multi-modal Transformer for Video Groundingcited by 54
  37. 2021End-to-end Multi-modal Video Temporal Groundingcited by 47
  38. 2020Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videoscited by 46

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 27; 2025: 30; 2026: 18.

79 papers.

Sort:

By area:

  1. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 146
  2. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
  3. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  4. 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
  5. 2024Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrievalcited by 30
  6. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  7. 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Modelscited by 28
  8. 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 27
  9. 2024SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Groundingcited by 25
  10. 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25
  11. 2024FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Groundingcited by 23
  12. 2024Saliency-Guided DETR for Moment Retrieval and Highlight Detectioncited by 21
38 more
  1. 2024VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPTcited by 19
  2. 2024Chrono: A Simple Blueprint for Representing Time in MLLMscited by 18
  3. 2024Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motioncited by 16
  4. 2023RGNet: A Unified Clip Retrieval and Grounding Network for Long Videoscited by 15
  5. 2025TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMscited by 15
  6. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
  7. 2024Improving Visual Recognition with Hyperbolical Visual Hierarchy Mappingcited by 11
  8. 2024Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Groundingcited by 10
  9. 2024Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Groundingcited by 10
  10. 2025Moment of Untruth: Dealing with Negative Queries in Video Moment Retrievalcited by 10
  11. 2025MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learningcited by 10
  12. 2025Tempo-R0: A Video-MLLM for Temporal Video Grounding through Efficient Temporal Sensing Reinforcement Learningcited by 9
  13. 2025LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detectioncited by 8
  14. 2025Prototypes are Balanced Units for Efficient and Effective Partially Relevant Video Retrievalcited by 8
  15. 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learningcited by 8
  16. 2024Let Me Finish My Sentence: Video Temporal Grounding with Holistic Text Understandingcited by 7
  17. 2025TAG: A Simple Yet Effective Temporal-Aware Approach for Zero-Shot Video Temporal Groundingcited by 7
  18. 2025VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotationscited by 7
  19. 2024Unleash the Potential of CLIP for Video Highlight Detectioncited by 6
  20. 2024Diversifying Query: Region-Guided Transformer for Temporal Sentence Groundingcited by 6
  21. 2024Localizing Events in Videos with Multimodal Queriescited by 6
  22. 2024Lighthouse: A User-Friendly Library for Reproducible Video Moment Retrieval and Highlight Detectioncited by 5
  23. 2025TimeScope: Towards Task-Oriented Temporal Grounding In Long Videoscited by 5
  24. 2024Temporally Grounding Instructional Diagrams in Unconstrained Videoscited by 4
  25. 2024SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopsescited by 4
  26. 2025Moment Quantization for Video Temporal Groundingcited by 4
  27. 2025Sim-DETR: Unlock DETR for Temporal Sentence Groundingcited by 4
  28. 2025Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Groundingcited by 4
  29. 2026ArrowGEV: Grounding Events in Video via Learning the Arrow of Timecited by 4
  30. 2026CVA: Context-aware Video-text Alignment for Video Temporal Groundingcited by 4
  31. 2024SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Groundingcited by 3
  32. 2024VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrievalcited by 3
  33. 2025The Devil is in the Spurious Correlations: Boosting Moment Retrieval with Dynamic Learningcited by 3
  34. 2025CFSum: A Transformer-Based Multi-Modal Video Summarization Framework With Coarse-Fine Fusioncited by 3
  35. 2025TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videoscited by 3
  36. 2025When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactionscited by 3
  37. 2025CASTELLA: Long Audio Dataset with Captions and Temporal Boundariescited by 3
  38. 2025Learning to Refuse: Refusal-Aware Reinforcement Fine-Tuning for Hard-Irrelevant Queries in Video Temporal Groundingcited by 3

50 most cited of 79 All 79 (JSON)