mlsanityData to 7 Oct 2026

Lightweight Attentional Feature Fusion: A New Baseline for Text-to-Video Retrieval

arXiv 2112.01832 ↗First posted 3 Dec 2021cs.MM cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

In this paper we revisit feature fusion, an old-fashioned topic, in the new context of text-to-video retrieval. Different from previous research that considers feature fusion only at one end, let it be video or text, we aim for feature fusion for both ends within a unified framework. We hypothesize that optimizing the convex combination of the features is preferred to modeling their correlations by computationally heavy multi-head self attention. We propose Lightweight Attentional Feature Fusion (LAFF). LAFF performs feature fusion at both early and late stages and at both video and text ends, making it a powerful method for exploiting diverse (off-the-shelf) features. The interpretability of LAFF can be used for feature selection. Extensive experiments on five public benchmark sets (MSR-VTT, MSVD, TGIF, VATEX and TRECVID AVS 2016-2020) justify LAFF as a new baseline for text-to-video retrieval.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 3 Dec 2021.

Builds on

33 of 68 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  6. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  8. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  9. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  10. 2016Language Modeling with Gated Convolutional Networkscited by 688
  11. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  12. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
21 more
  1. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  2. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  3. 2020Multi-modal Transformer for Video Retrievalcited by 395
  4. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  5. 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
  6. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  7. 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
  8. 2020Support-set bottlenecks for video-text representation learningcited by 170
  9. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  10. 2019Large-scale weakly-supervised pre-training for video action recognitioncited by 130
  11. 2016TGIF: A New Dataset and Benchmark on Animated GIF Descriptioncited by 124
  12. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  13. 2019Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddingscited by 101
  14. 2019Polysemous Visual-Semantic Embedding for Cross-Modal Retrievalcited by 83
  15. 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
  16. 2020Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learningcited by 74
  17. 2021A Straightforward Framework For Video Retrieval Using CLIPcited by 66
  18. 2017Predicting Visual Features from Text for Image and Video Caption Retrievalcited by 53
  19. 2020Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrievalcited by 47
  20. 2020SEA: Sentence Encoder Assembly for Video Retrieval by Textual Queriescited by 26
  21. 2021What Matters for Ad-hoc Video Search? A Large-scale Evaluation on TRECVIDcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 5; 2024: 6; 2025: 4; 2026: 4.

23 papers.

Sort:

By area:

  1. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  2. 2023A Review of Deep Learning for Video Captioningcited by 28
  3. 2023Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignmentcited by 21
  4. 2022Learn to Understand Negation in Video Retrievalcited by 16
  5. 2024Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learningcited by 12
  6. 2023CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfercited by 10
  7. 2022PRVR: Partially Relevant Video Retrievalcited by 7
  8. 2023Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrievalcited by 5
  9. 2025Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrievalcited by 4
  10. 2022(Un)likelihood Training for Interpretable Embeddingcited by 3
  11. 2024Improving Interpretable Embeddings for Ad-hoc Video Search with Generative Captions and Multi-word Concept Bankcited by 3
  12. 2025Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrievalcited by 3
11 more
  1. 2022Renmin University of China at TRECVID 2022: Improving Video Search by Feature Fusion and Negation Understandingcited by 1
  2. 2023TeachCLIP: Multi-Grained Teaching for Efficient Text-to-Video Retrievalcited by 1
  3. 2024Video Editing for Video Retrievalcited by 1
  4. 2024Multimodal LLM-based Query Paraphrasing for Video Searchcited by 1
  5. 2024Beyond Coarse-Grained Matching in Video-Text Retrievalcited by 1
  6. 2025Leveraging Auxiliary Information in Text-to-Video Retrieval: A Reviewcited by 1
  7. 2026SpatialMem: Metric-Aligned Long-Horizon Video Memory for Language Grounding and QAcited by 1
  8. 2025Adaptive Multi-Agent Reasoning for Text-to-Video Retrievalcited by 0
  9. 2026SAVE: Speech-Aware Video Representation Learning for Video-Text Retrievalcited by 0
  10. 2026Fusion Learning from Dynamic Functional Connectivity: Combining the Amplitude and Phase of fMRI Signals to Identify Brain Disorderscited by 0
  11. 2026EagleNet: Energy-Aware Fine-Grained Relationship Learning Network for Text-Video Retrievalcited by 0