mlsanityData to 7 Oct 2026

Multilevel Language and Vision Integration for Text-to-Clip Retrieval

arXiv 1804.05113 ↗First posted 13 Apr 2018cs.CVcited by 128

Not on the map. Why?

Abstract (arXiv)

We address the problem of text-based activity retrieval in video. Given a sentence describing an activity, our task is to retrieve matching clips from an untrimmed video. To capture the inherent structures present in both text and video, we introduce a multilevel model that integrates vision and language features earlier and more tightly than prior work. First, we inject text features early on when generating clip proposals, to help eliminate unlikely clips and thus speed up processing and boost performance. Second, to learn a fine-grained similarity metric for retrieval, we use visual features to modulate the processing of query sentences at the word level in a recurrent neural network. A multi-task loss is also employed by adding query re-generation as an auxiliary task. Our approach significantly outperforms prior work on two challenging benchmarks: Charades-STA and ActivityNet Captions.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 22; 2023: 27; 2024: 17; 2025: 17; 2026: 9.

128 papers.

Sort:

By area:

  1. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  2. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
  3. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
  4. 2021Negative Sample Matters: A Renaissance of Metric Learning for Temporal Groundingcited by 90
  5. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  6. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  7. 2021QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queriescited by 72
  8. 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
  9. 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
  10. 2021Natural Language Video Localization: A Revisit in Span-based Question Answering Frameworkcited by 55
  11. 2021On Pursuit of Designing Multi-modal Transformer for Video Groundingcited by 54
  12. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
38 more
  1. 2021End-to-end Multi-modal Video Temporal Groundingcited by 47
  2. 2022Memory-Guided Semantic Learning Network for Temporal Sentence Groundingcited by 46
  3. 2022LocVTP: Video-Text Pre-training for Temporal Localizationcited by 44
  4. 2022Multi-Modal Cross-Domain Alignment Network for Video Moment Retrievalcited by 43
  5. 2021A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metriccited by 42
  6. 2021Adaptive Proposal Generation Network for Temporal Sentence Localization in Videoscited by 42
  7. 2023MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformercited by 40
  8. 2021Cross-Sentence Temporal and Semantic Relations in Video Activity Localisationcited by 39
  9. 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
  10. 2021Weakly Supervised Temporal Adjacent Network for Language Groundingcited by 37
  11. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  12. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  13. 2024SnAG: Scalable and Accurate Video Groundingcited by 36
  14. 2022Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Groundingcited by 35
  15. 2022Reducing the Vision and Language Bias for Temporal Sentence Groundingcited by 35
  16. 2021Read and Attend: Temporal Localisation in Sign Language Videoscited by 34
  17. 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
  18. 2024Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrievalcited by 30
  19. 2021Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Groundingcited by 29
  20. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  21. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  22. 2021Support-Set Based Cross-Supervision for Video Groundingcited by 28
  23. 2021Natural Language Video Localization with Learnable Moment Proposalscited by 27
  24. 2021Relation-aware Video Reading Comprehension for Temporal Language Groundingcited by 27
  25. 2022CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Groundingcited by 27
  26. 2021Aligning Subtitles in Sign Language Videoscited by 26
  27. 2022Exploring Motion and Appearance Information for Temporal Sentence Groundingcited by 24
  28. 2021Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphscited by 23
  29. 2024FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Groundingcited by 23
  30. 2021Parallel Attention Network with Sequence Matching for Video Groundingcited by 22
  31. 2022Can Shuffling Video Benefit Temporal Bias Problem: A Novel Training Framework for Temporal Groundingcited by 22
  32. 2023ViGT: Proposal-free Video Grounding with Learnable Token in Transformercited by 22
  33. 2023Zero-Shot Video Moment Retrieval from Frozen Vision-Language Modelscited by 22
  34. 2022Skimming, Locating, then Perusing: A Human-Like Framework for Natural Language Video Localizationcited by 21
  35. 2023G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theorycited by 21
  36. 2022Video Moment Retrieval from Text Queries via Single Frame Annotationcited by 19
  37. 2023Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Groundingcited by 19
  38. 2023Text-Visual Prompting for Efficient 2D Temporal Video Groundingcited by 18

50 most cited of 128 All 128 (JSON)