mlsanityData to 14 Oct 2026

MRTNet: Multi-Resolution Temporal Network for Video Sentence Grounding

arXiv 2212.13163 ↗First posted 26 Dec 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Given an untrimmed video and natural language query, video sentence grounding aims to localize the target temporal moment in the video. Existing methods mainly tackle this task by matching and aligning semantics of the descriptive sentence and video segments on a single temporal resolution, while neglecting the temporal consistency of video content in different resolutions. In this work, we propose a novel multi-resolution temporal video sentence grounding network: MRTNet, which consists of a multi-modal feature encoder, a Multi-Resolution Temporal (MRT) module, and a predictor module. MRT module is an encoder-decoder network, and output features in the decoder part are in conjunction with Transformers to predict the final start and end timestamps. Particularly, our MRT module is hot-pluggable, which means it can be seamlessly incorporated into any anchor-free models. Besides, we utilize a hybrid loss to supervise cross-modal features in MRT module for more accurate grounding in three scales: frame-level, clip-level and sequence-level. Extensive experiments on three prevalent datasets have shown the effectiveness of MRTNet.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 40 of its 68 references that are in the map. First posted on arXiv 26 Dec 2022.

Builds on

40 of 68 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  7. 2017Localizing Moments in Video with Natural Languagecited by 663
  8. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  9. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
  10. 2020Multi-modal Transformer for Video Retrievalcited by 395
  11. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  12. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
28 more
  1. 2020Local-Global Video-Text Interactions for Temporal Groundingcited by 188
  2. 2020Dense Regression Network for Video Groundingcited by 176
  3. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  4. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  5. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  6. 2019Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videoscited by 135
  7. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  8. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
  9. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  10. 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 87
  11. 2019Multimodal Transformer with Multi-View Visual Representation for Image Captioningcited by 86
  12. 2018MAC: Mining Activity Concepts for Language-based Temporal Localizationcited by 84
  13. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  14. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  15. 2018Localizing Moments in Video with Temporal Languagecited by 83
  16. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  17. 2019Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videoscited by 79
  18. 2020Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localizationcited by 74
  19. 2021Video as Conditional Graph Hierarchy for Multi-Granular Question Answeringcited by 68
  20. 2022Invariant Grounding for Video Question Answeringcited by 68
  21. 2019Read, Watch, and Move: Reinforcement Learning for Temporally Grounding Natural Language Descriptions in Videoscited by 60
  22. 2022Multi-Modal Cross-Domain Alignment Network for Video Moment Retrievalcited by 43
  23. 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
  24. 2018Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architecturescited by 30
  25. 2022RBC: Rectifying the Biased Context in Continual Semantic Segmentationcited by 27
  26. 2021Parallel Attention Network with Sequence Matching for Video Groundingcited by 22
  27. 2022Entity-aware and Motion-aware Transformers for Language-driven Action Localization in Videoscited by 10
  28. 2022Learning Commonsense-aware Moment-Text Alignment for Fast Video Temporal Groundingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 0; 2026: 2.

6 papers.

  1. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 95
  2. 2023Generating Visual Scenes from Touchcited by 26
  3. 2024Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Networkcited by 15
  4. 2026Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Languagecited by 6
  5. 2026Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Languagecited by 5
  6. 2024Scalable Frame-based Construction of Sociocultural NormBases for Socially-Aware Dialoguescited by 1