mlsanityData to 7 Oct 2026

Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach

arXiv 2408.07249 ↗First posted 14 Aug 2024cs.CV cs.IRcited by 5

Not on the map. Why?

Abstract (arXiv)

As online video content rapidly grows, the task of text-video retrieval (TVR) becomes increasingly important. A key challenge in TVR is the information asymmetry between video and text: videos are inherently richer in information, while their textual descriptions often capture only fragments of this complexity. This paper introduces a novel, data-centric framework to bridge this gap by enriching textual representations to better match the richness of video content. During training, videos are segmented into event-level clips and captioned to ensure comprehensive coverage. During retrieval, a large language model (LLM) generates semantically diverse queries to capture a broader range of possible matches. To enhance retrieval efficiency, we propose a query selection mechanism that identifies the most relevant and diverse queries, reducing computational cost while improving accuracy. Our method achieves state-of-the-art results across multiple benchmarks, demonstrating the power of data-centric approaches in addressing information asymmetry in TVR. This work paves the way for new research focused on leveraging data to improve cross-modal retrieval.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 54 of its 73 references that are in the map. First posted on arXiv 14 Aug 2024.

Builds on

54 of 73 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  11. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  12. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
38 more
  1. 2022Efficiently Scaling Transformer Inferencecited by 529
  2. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  3. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  4. 2020Multi-modal Transformer for Video Retrievalcited by 395
  5. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  6. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  7. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  8. 2023Improving CLIP Training with Language Rewritescited by 216
  9. 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
  10. 2023Video Understanding with Large Language Models: A Surveycited by 203
  11. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  12. 2015A Dataset for Movie Descriptioncited by 195
  13. 2023Multimodal Large Language Models: A Surveycited by 193
  14. 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
  15. 2020Support-set bottlenecks for video-text representation learningcited by 170
  16. 2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
  17. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  18. 2022X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrievalcited by 127
  19. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  20. 2022TS2-Net: Token Shift and Selection Transformer for Text-Video Retrievalcited by 104
  21. 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
  22. 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
  23. 2022CenterCLIP: Token Clustering for Efficient Text-Video Retrievalcited by 86
  24. 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
  25. 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
  26. 2022Disentangled Representation Learning for Text-Video Retrievalcited by 77
  27. 2020Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learningcited by 74
  28. 2021Cross Modal Retrieval with Querybank Normalisationcited by 71
  29. 2021A Straightforward Framework For Video Retrieval Using CLIPcited by 66
  30. 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
  31. 2023UATVR: Uncertainty-Adaptive Text-Video Retrievalcited by 55
  32. 2023Unified Coarse-to-Fine Alignment for Video-Text Retrievalcited by 49
  33. 2020BERT-QE: Contextualized Query Expansion for Document Re-rankingcited by 40
  34. 2024Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrievalcited by 33
  35. 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
  36. 2016Deep Learning for Video Classification and Captioningcited by 28
  37. 2020Show, Recall, and Tell: Image Captioning with Recall Mechanismcited by 27
  38. 2023Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignmentcited by 21

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.