Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach
Not on the map. Why?
Abstract (arXiv)
As online video content rapidly grows, the task of text-video retrieval (TVR) becomes increasingly important. A key challenge in TVR is the information asymmetry between video and text: videos are inherently richer in information, while their textual descriptions often capture only fragments of this complexity. This paper introduces a novel, data-centric framework to bridge this gap by enriching textual representations to better match the richness of video content. During training, videos are segmented into event-level clips and captioned to ensure comprehensive coverage. During retrieval, a large language model (LLM) generates semantically diverse queries to capture a broader range of possible matches. To enhance retrieval efficiency, we propose a query selection mechanism that identifies the most relevant and diverse queries, reducing computational cost while improving accuracy. Our method achieves state-of-the-art results across multiple benchmarks, demonstrating the power of data-centric approaches in addressing information asymmetry in TVR. This work paves the way for new research focused on leveraging data to improve cross-modal retrieval.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 54 of its 73 references that are in the map. First posted on arXiv 14 Aug 2024.
Builds on
54 of 73 references.
References · 54
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
38 more
- 2022Efficiently Scaling Transformer Inferencecited by 529
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
- 2020Multi-modal Transformer for Video Retrievalcited by 395
- 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
- 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
- 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
- 2023Improving CLIP Training with Language Rewritescited by 216
- 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
- 2023Video Understanding with Large Language Models: A Surveycited by 203
- 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
- 2015A Dataset for Movie Descriptioncited by 195
- 2023Multimodal Large Language Models: A Surveycited by 193
- 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
- 2020Support-set bottlenecks for video-text representation learningcited by 170
- 2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
- 2023Query2doc: Query Expansion with Large Language Modelscited by 146
- 2022X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrievalcited by 127
- 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
- 2022TS2-Net: Token Shift and Selection Transformer for Text-Video Retrievalcited by 104
- 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
- 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
- 2022CenterCLIP: Token Clustering for Efficient Text-Video Retrievalcited by 86
- 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
- 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
- 2022Disentangled Representation Learning for Text-Video Retrievalcited by 77
- 2020Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learningcited by 74
- 2021Cross Modal Retrieval with Querybank Normalisationcited by 71
- 2021A Straightforward Framework For Video Retrieval Using CLIPcited by 66
- 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
- 2023UATVR: Uncertainty-Adaptive Text-Video Retrievalcited by 55
- 2023Unified Coarse-to-Fine Alignment for Video-Text Retrievalcited by 49
- 2020BERT-QE: Contextualized Query Expansion for Document Re-rankingcited by 40
- 2024Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrievalcited by 33
- 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
- 2016Deep Learning for Video Classification and Captioningcited by 28
- 2020Show, Recall, and Tell: Image Captioning with Recall Mechanismcited by 27
- 2023Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignmentcited by 21
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025Think Then Embed: Generative Context Improves Multimodal Embeddingcited by 39
- 2025Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchorscited by 4
- 2025Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videoscited by 1
- 2025Leveraging Auxiliary Information in Text-to-Video Retrieval: A Reviewcited by 1
- 2026Robust Test-time Video-Text Retrieval: Benchmarking and Adapting for Query Shiftscited by 0
- 2026Robust Test-time Video-Text Retrieval: Benchmarking and Adapting for Query Shifts15 Feb 2026
- 2025Think Then Embed: Generative Context Improves Multimodal Embedding6 Oct 2025
- 2025Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchors26 Sep 2025
- 2025Leveraging Auxiliary Information in Text-to-Video Retrieval: A Review29 May 2025
- 2025Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos8 May 2025