mlsanityData to 7 Oct 2026

A CLIP-Hitchhiker's Guide to Long Video Retrieval

arXiv 2205.08508 ↗First posted 17 May 2022cs.CVcited by 57

Not on the map. Why?

Abstract (arXiv)

Our goal in this paper is the adaptation of image-text models for long video retrieval. Recent works have demonstrated state-of-the-art performance in video retrieval by adopting CLIP, effectively hitchhiking on the image-text representation for video tasks. However, there has been limited success in learning temporal aggregation that outperform mean-pooling the image-level representations extracted per frame by CLIP. We find that the simple yet effective baseline of weighted-mean of frame embeddings via query-scoring is a significant improvement above all prior temporal modelling attempts and mean-pooling. In doing so, we provide an improved baseline for others to compare to and demonstrate state-of-the-art performance of this simple baseline on a suite of long video retrieval benchmarks.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 55 of its 65 references that are in the map. First posted on arXiv 17 May 2022.

Builds on

55 of 65 references.

References · 55

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  10. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  12. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
38 more
  1. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  2. 2021Scaling Vision Transformerscited by 993
  3. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  4. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  5. 2017Dense-Captioning Events in Videoscited by 789
  6. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  7. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  8. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  9. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  10. 2020Multi-modal Transformer for Video Retrievalcited by 395
  11. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  12. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  13. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  14. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  15. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  16. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  17. 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
  18. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  19. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  20. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  21. 2020Support-set bottlenecks for video-text representation learningcited by 170
  22. 2021Towards Long-Form Video Understandingcited by 146
  23. 2015Learning Deep Structure-Preserving Image-Text Embeddingscited by 139
  24. 2019SCSampler: Sampling Salient Clips from Video for Efficient Action Recognitioncited by 133
  25. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  26. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  27. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  28. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  29. 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
  30. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  31. 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
  32. 2019Weakly Supervised Video Moment Retrieval From Text Queriescited by 85
  33. 2016Long-term Temporal Convolutions for Action Recognitioncited by 84
  34. 2022Temporal Alignment Networks for Long-term Videocited by 84
  35. 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
  36. 2021Everything at Once -- Multi-modal Fusion Transformer for Video Retrievalcited by 79
  37. 2021Cross Modal Retrieval with Querybank Normalisationcited by 71
  38. 2017Learnable pooling with Context Gating for video classificationcited by 58

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 20; 2024: 18; 2025: 8; 2026: 3.

57 papers.

Sort:

By area:

  1. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 118
  2. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  3. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 102
  4. 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
  5. 2022VindLU: A Recipe for Effective Video-and-Language Pretrainingcited by 87
  6. 2022VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captionerscited by 69
  7. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  8. 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
  9. 2023VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Modelscited by 55
  10. 2023CoVR-2: Automatic Data Construction for Composed Video Retrievalcited by 52
  11. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  12. 2023AutoAD: Movie Description in Contextcited by 50
38 more
  1. 2023Unified Coarse-to-Fine Alignment for Video-Text Retrievalcited by 49
  2. 2023TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understandingcited by 49
  3. 2023UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modelingcited by 40
  4. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  5. 2022Cross-Modal Adapter for Vision-Language Retrievalcited by 35
  6. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  7. 2024Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrievalcited by 33
  8. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  9. 2023Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignmentcited by 31
  10. 2023EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognitioncited by 31
  11. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  12. 2024DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrievalcited by 26
  13. 2023Parameter-efficient Tuning of Large-scale Multimodal Foundation Modelcited by 23
  14. 2024EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrievalcited by 21
  15. 2025CoLLM: A Large Language Model for Composed Image Retrievalcited by 21
  16. 2022Video + CLIP Baseline for Ego4D Long-term Action Anticipationcited by 18
  17. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Roboticscited by 17
  18. 2024Learning Video Context as Interleaved Multimodal Sequencescited by 16
  19. 2025MADGEN: Mass-Spec attends to De Novo Molecular generationcited by 16
  20. 2023Language-based Action Concept Spaces Improve Video Self-Supervised Learningcited by 10
  21. 2025ViLU: Learning Vision-Language Uncertainties for Failure Predictioncited by 9
  22. 2023Actor-agnostic Multi-label Action Recognition with Multi-modal Querycited by 8
  23. 2024MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learningcited by 8
  24. 2022PRVR: Partially Relevant Video Retrievalcited by 7
  25. 2023ProbVLM: Probabilistic Adapter for Frozen Vision-Language Modelscited by 7
  26. 2023MultiVENT: Multilingual Videos of Events with Aligned Natural Textcited by 7
  27. 2024Detours for Navigating Instructional Videoscited by 7
  28. 2024Learning text-to-video retrieval from image captioningcited by 6
  29. 2024Finetuning CLIP to Reason about Pairwise Differencescited by 6
  30. 2025Beyond Simple Edits: Composed Video Retrieval with Dense Modificationscited by 6
  31. 2024SRL-CLIP: Efficient CLIP Video Adaptation via Structured Semantic Role Labelscited by 4
  32. 2022VLG: General Video Recognition with Web Textual Knowledgecited by 3
  33. 2024Dissecting Temporal Understanding in Text-to-Audio Retrievalcited by 3
  34. 2025Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrievalcited by 3
  35. 2024Multi-modal News Understanding with Professionally Labelled Videos (ReutersViLNews)cited by 2
  36. 2025Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Surveycited by 2
  37. 2023Is a Video worth n× n Images? A Highly Efficient Approach to Transformer-based Video Question Answeringcited by 1
  38. 2023AAN: Attributes-Aware Network for Temporal Action Detectioncited by 1

50 most cited of 57 All 57 (JSON)