mlsanityData to 14 Oct 2026

Sim-DETR: Unlock DETR for Temporal Sentence Grounding

arXiv 2509.23867 ↗First posted 28 Sep 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Temporal sentence grounding aims to identify exact moments in a video that correspond to a given textual query, typically addressed with detection transformer (DETR) solutions. However, we find that typical strategies designed to enhance DETR do not improve, and may even degrade, its performance in this task. We systematically analyze and identify the root causes of this abnormal behavior: (1) conflicts between queries from similar target moments and (2) internal query conflicts due to the tension between global semantics and local localization. Building on these insights, we propose a simple yet powerful baseline, Sim-DETR, which extends the standard DETR with two minor modifications in the decoder layers: (1) constraining self-attention between queries based on their semantic and positional overlap and (2) adding query-to-frame alignment to bridge the global and local contexts. Experiments demonstrate that Sim-DETR unlocks the full potential of DETR for temporal sentence grounding, offering a strong baseline for future research.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 61 of its 84 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

61 of 84 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  10. 2017Localizing Moments in Video with Natural Languagecited by 663
  11. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  12. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
38 more
  1. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  2. 2021Conditional DETR for Fast Training Convergencecited by 424
  3. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  4. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  5. 2020Self-supervised Co-training for Video Representation Learningcited by 232
  6. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
  7. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  8. 2020Dense Regression Network for Video Groundingcited by 176
  9. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  10. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  11. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  12. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  13. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  14. 2019Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videoscited by 135
  15. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  16. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  17. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 128
  18. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  19. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  20. 2018MAC: Mining Activity Concepts for Language-based Temporal Localizationcited by 84
  21. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  22. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  23. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  24. 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
  25. 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
  26. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  27. 2021PnP-DETR: Towards Efficient Visual Analysis with Transformerscited by 52
  28. 2022Memory-Guided Semantic Learning Network for Temporal Sentence Groundingcited by 46
  29. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignmentcited by 44
  30. 2021Adaptive Proposal Generation Network for Temporal Sentence Localization in Videoscited by 42
  31. 2022Reducing the Vision and Language Bias for Temporal Sentence Groundingcited by 35
  32. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  33. 2019Exploiting Temporal Relationships in Video Moment Localization with Natural Languagecited by 33
  34. 2023LoGoPrompt: Synthetic Text Images Can Be Good Visual Prompts for Vision-Language Modelscited by 29
  35. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  36. 2023Temporal Collection and Distribution for Referring Video Object Segmentationcited by 27
  37. 2024SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Groundingcited by 25
  38. 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026RefAny3D: 3D Asset-Referenced Diffusion Models for Image Generationcited by 2
  2. 2025Augmenting Moment Retrieval: Zero-Dependency Two-Stage Learningcited by 1
  3. 2026Beyond Caption-Based Queries for Video Moment Retrievalcited by 0
  4. 2026D'ej`a Cue: Localizing States in Object Histories via Vocabulary-Relative Coordinatescited by 0