mlsanityData to 14 Oct 2026

MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction

arXiv 2305.18969 ↗First posted 30 May 2023cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Given a query, the task of Natural Language Video Localization (NLVL) is to localize a temporal moment in an untrimmed video that semantically matches the query. In this paper, we adopt a proposal-based solution that generates proposals (i.e., candidate moments) and then select the best matching proposal. On top of modeling the cross-modal interaction between candidate moments and the query, our proposed Moment Sampling DETR (MS-DETR) enables efficient moment-moment relation modeling. The core idea is to sample a subset of moments guided by the learnable templates with an adopted DETR (DEtection TRansformer) framework. To achieve this, we design a multi-scale visual-linguistic encoder, and an anchor-guided moment decoder paired with a set of learnable templates. Experimental results on three public datasets demonstrate the superior performance of MS-DETR.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

34 of 49 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020End-to-End Object Detection with Transformerscited by 7,414
  3. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  4. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  5. 2017Dense-Captioning Events in Videoscited by 793
  6. 2017Localizing Moments in Video with Natural Languagecited by 663
  7. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  8. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
  9. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  10. 2021Conditional DETR for Fast Training Convergencecited by 424
  11. 2020Multi-modal Transformer for Video Retrievalcited by 395
  12. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 303
22 more
  1. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  2. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  3. 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
  4. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  5. 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
  6. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  7. 2022DETRs with Hybrid Matchingcited by 150
  8. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  9. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
  10. 2022Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignmentcited by 98
  11. 2021Negative Sample Matters: A Renaissance of Metric Learning for Temporal Groundingcited by 90
  12. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  13. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  14. 2021Natural Language Video Localization: A Revisit in Span-based Question Answering Frameworkcited by 55
  15. 2021On Pursuit of Designing Multi-modal Transformer for Video Groundingcited by 54
  16. 2019Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systemscited by 52
  17. 2021Adaptive Proposal Generation Network for Temporal Sentence Localization in Videoscited by 42
  18. 2020Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 29
  19. 2021Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Groundingcited by 29
  20. 2021Natural Language Video Localization with Learnable Moment Proposalscited by 27
  21. 2021Parallel Attention Network with Sequence Matching for Video Groundingcited by 22
  22. 2021Structured Co-reference Graph Attention for Video-grounded Dialoguecited by 17

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 4.

8 papers.

  1. 2023Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrievalcited by 27
  2. 2025Towards Efficient Partially Relevant Video Retrieval with Active Moment Discoveringcited by 18
  3. 2024Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Groundingcited by 12
  4. 2025LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detectioncited by 8
  5. 2024Temporally Grounding Instructional Diagrams in Unconstrained Videoscited by 4
  6. 2024MLP: Motion Label Prior for Temporal Sentence Localization in Untrimmed 3D Human Motionscited by 3
  7. 2025Audio Does Matter: Importance-Aware Multi-Granularity Fusion for Video Moment Retrievalcited by 2
  8. 2025Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localizationcited by 0