Leveraging Auxiliary Information in Text-to-Video Retrieval: A Review
Not on the map. Why?
Abstract (arXiv)
Text-to-Video (T2V) retrieval aims to identify the most relevant item from a gallery of videos based on a user's text query. Traditional methods rely solely on aligning video and text modalities to compute the similarity and retrieve relevant items. However, recent advancements emphasise incorporating auxiliary information extracted from video and text modalities to improve retrieval performance and bridge the semantic gap between these modalities. Auxiliary information can include visual attributes, such as objects; temporal and spatial context; and textual descriptions, such as speech and rephrased captions. This survey comprehensively reviews 81 research papers on Text-to-Video retrieval that utilise such auxiliary information. It provides a detailed analysis of their methodologies; highlights state-of-the-art results on benchmark datasets; and discusses available datasets and their auxiliary information. Additionally, it proposes promising directions for future research, focusing on different ways to further enhance retrieval performance using this information.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 81 of its 112 references that are in the map. First posted on arXiv 29 May 2025.
Builds on
81 of 112 references.
References · 81
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
- 2017Dense-Captioning Events in Videoscited by 793
- 2017Localizing Moments in Video with Natural Languagecited by 663
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
- 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
- 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
38 more
- 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
- 2020Multi-modal Transformer for Video Retrievalcited by 395
- 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
- 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
- 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
- 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
- 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
- 2015A Dataset for Movie Descriptioncited by 195
- 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
- 2020Support-set bottlenecks for video-text representation learningcited by 170
- 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 169
- 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
- 2022X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrievalcited by 128
- 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
- 2020AVLnet: Learning Audio-Visual Language Representations from Instructional Videoscited by 111
- 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
- 2018Cross-Modal and Hierarchical Modeling of Video and Textcited by 103
- 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
- 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
- 2019Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddingscited by 101
- 2020COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learningcited by 100
- 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
- 2021HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrievalcited by 87
- 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
- 2022Learning Audio-Video Modalities from Image Captionscited by 80
- 2021Everything at Once -- Multi-modal Fusion Transformer for Video Retrievalcited by 79
- 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
- 2022CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignmentcited by 74
- 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
- 2016A Comprehensive Survey on Cross-modal Retrievalcited by 62
- 2021Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videoscited by 62
- 2016End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answeringcited by 60
- 2021Object-aware Video-language Pre-training for Retrievalcited by 57
- 2024Gramian Multimodal Representation Learning and Alignmentcited by 54
- 2020Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrievalcited by 47
- 2024Foundation Models for Video Understanding: A Surveycited by 46
- 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
- 2022ECLIPSE: Efficient Long-range Video Retrieval using Sight and Soundcited by 39
50 most cited of 81 All 81 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.