mlsanityData to 7 Oct 2026

Use What You Have: Video Retrieval Using Representations From Collaborative Experts

arXiv 1907.13487 ↗First posted 31 Jul 2019cs.CVcited by 214

Not on the map. Why?

Abstract (arXiv)

The rapid growth of video on the internet has made searching for video content using natural language queries a significant challenge. Human-generated queries for video datasets `in the wild' vary a lot in terms of degree of specificity, with some queries describing specific details such as the names of famous identities, content from speech, or text available on the screen. Our goal is to condense the multi-modal, extremely high dimensional information from videos into a single, compact video representation for the task of video retrieval using free-form text queries, where the degree of specificity is open-ended. For this we exploit existing knowledge in the form of pre-trained semantic embeddings which include 'general' features such as motion, appearance, and scene features from visual content. We also explore the use of more 'specific' cues from ASR and OCR which are intermittently available for videos and find that these signals remain challenging to use effectively for retrieval. We propose a collaborative experts model to aggregate information from these different pre-trained experts and assess our approach empirically on five retrieval benchmarks: MSR-VTT, LSMDC, MSVD, DiDeMo, and ActivityNet. Code and data can be found at www.robots.ox.ac.uk/~vgg/research/collaborative-experts/. This paper contains a correction to results reported in the previous version.

Key facts. Cited by 214 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 76; 2023: 39; 2024: 27; 2025: 20; 2026: 8.

214 papers.

Sort:

By area:

  1. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  2. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  3. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  4. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  5. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  6. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  7. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  8. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  9. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  10. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  11. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  12. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
38 more
  1. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  2. 2022Revealing Single Frame Bias for Video-and-Language Learningcited by 131
  3. 2022X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrievalcited by 127
  4. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  5. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 110
  6. 2022TS2-Net: Token Shift and Selection Transformer for Text-Video Retrievalcited by 104
  7. 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
  8. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
  9. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  10. 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
  11. 2021HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrievalcited by 87
  12. 2022VindLU: A Recipe for Effective Video-and-Language Pretrainingcited by 87
  13. 2022CenterCLIP: Token Clustering for Efficient Text-Video Retrievalcited by 86
  14. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  15. 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
  16. 2022Learning Audio-Video Modalities from Image Captionscited by 80
  17. 2022Bridging Video-text Retrieval with Multiple Choice Questionscited by 79
  18. 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
  19. 2022Disentangled Representation Learning for Text-Video Retrievalcited by 77
  20. 2022An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modelingcited by 74
  21. 2022CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignmentcited by 74
  22. 2021CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representationscited by 72
  23. 2021Cross Modal Retrieval with Querybank Normalisationcited by 71
  24. 2021Temporal Query Networks for Fine-grained Video Understandingcited by 68
  25. 2021A Straightforward Framework For Video Retrieval Using CLIPcited by 66
  26. 2021Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videoscited by 62
  27. 2022Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learningcited by 59
  28. 2021Object-aware Video-language Pre-training for Retrievalcited by 57
  29. 2022A CLIP-Hitchhiker's Guide to Long Video Retrievalcited by 57
  30. 2022Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representationscited by 57
  31. 2023UATVR: Uncertainty-Adaptive Text-Video Retrievalcited by 55
  32. 2021Distilling Audio-Visual Knowledge by Compositional Contrastive Learningcited by 53
  33. 2023DiffusionRet: Generative Text-Video Retrieval with Diffusion Modelcited by 52
  34. 2022Towards Robust Referring Video Object Segmentation with Cyclic Relational Consensuscited by 49
  35. 2023Unified Coarse-to-Fine Alignment for Video-Text Retrievalcited by 49
  36. 2021On Semantic Similarity in Video Retrievalcited by 45
  37. 2023Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learningcited by 45
  38. 2022LocVTP: Video-Text Pre-training for Temporal Localizationcited by 44

50 most cited of 214 All 214 (JSON)