mlsanityData to 14 Oct 2026

Relation-aware Video Reading Comprehension for Temporal Language Grounding

arXiv 2110.05717 ↗First posted 12 Oct 2021cs.CV cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

Temporal language grounding in videos aims to localize the temporal span relevant to the given query sentence. Previous methods treat it either as a boundary regression task or a span extraction task. This paper will formulate temporal language grounding into video reading comprehension and propose a Relation-aware Network (RaNet) to address it. This framework aims to select a video moment choice from the predefined answer set with the aid of coarse-and-fine choice-query interaction and choice-choice relation construction. A choice-query interactor is proposed to match the visual and textual information simultaneously in sentence-moment and token-moment levels, leading to a coarse-and-fine cross-modal interaction. Moreover, a novel multi-choice relation constructor is introduced by leveraging graph convolution to capture the dependencies among video moment choices for the best choice selection. Extensive experiments on ActivityNet-Captions, TACoS, and Charades-STA demonstrate the effectiveness of our solution. Codes have been available.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 12 Oct 2021.

Builds on

30 of 44 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  7. 2017Dense-Captioning Events in Videoscited by 793
  8. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  9. 2017Localizing Moments in Video with Natural Languagecited by 663
  10. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  11. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
  12. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
18 more
  1. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  2. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 249
  3. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  4. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
  5. 2020Dense Regression Network for Video Groundingcited by 176
  6. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  7. 2019DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehensioncited by 156
  8. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  9. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  10. 2018Dual Encoding for Zero-Example Video Retrievalcited by 85
  11. 2018MAC: Mining Activity Concepts for Language-based Temporal Localizationcited by 84
  12. 2019Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Predictioncited by 84
  13. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  14. 2020Location-aware Graph Convolutional Networks for Video Question Answeringcited by 80
  15. 2019Read, Watch, and Move: Reinforcement Learning for Temporally Grounding Natural Language Descriptions in Videoscited by 60
  16. 2019Exploiting Temporal Relationships in Video Moment Localization with Natural Languagecited by 33
  17. 2020Accurate Temporal Action Proposal Generation with Relation-Aware Pyramid Networkcited by 28
  18. 2020Machine Reading Comprehension: The Role of Contextualized Language Models and Beyondcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 6; 2024: 7; 2025: 5; 2026: 3.

27 papers.

Sort:

By area:

  1. 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
  2. 2022Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Videocited by 43
  3. 2023You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videoscited by 41
  4. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  5. 2024SnAG: Scalable and Accurate Video Groundingcited by 36
  6. 2023Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrievalcited by 27
  7. 2022End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Groundingcited by 19
  8. 2023A Survey on Video Moment Localizationcited by 18
  9. 2022You Need to Read Again: Multi-granularity Perception Network for Moment Retrieval in Videoscited by 13
  10. 2024Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Groundingcited by 12
  11. 2023DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Groundingcited by 11
  12. 2025LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detectioncited by 8
15 more
  1. 2022Visual Answer Localization with Cross-modal Mutual Knowledge Transfercited by 7
  2. 2024Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Videocited by 7
  3. 2024Localizing Events in Videos with Multimodal Queriescited by 6
  4. 2026Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Languagecited by 6
  5. 2024Multi-Scale Contrastive Learning for Video Temporal Groundingcited by 5
  6. 2023Commonsense for Zero-Shot Natural Language Video Localizationcited by 4
  7. 2022Learning Commonsense-aware Moment-Text Alignment for Fast Video Temporal Groundingcited by 3
  8. 2025TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videoscited by 3
  9. 2024Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrievalcited by 2
  10. 2024ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Groundingcited by 2
  11. 2025HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Poolingcited by 1
  12. 2025LLM-powered Query Expansion for Enhancing Boundary Prediction in Language-driven Action Localizationcited by 0
  13. 2025Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrievalcited by 0
  14. 2026A Paradigm Shift: Fully End-to-End Training for Temporal Sentence Grounding in Videoscited by 0
  15. 2026Static and Dynamic Graph Alignment Network for Temporal Video Groundingcited by 0