mlsanityData to 7 Oct 2026

BagFormer: Better Cross-Modal Retrieval via bag-wise interaction

arXiv 2212.14322 ↗First posted 29 Dec 2022cs.IR cs.AI cs.MMcited by 1

Not on the map. Why?

Abstract (arXiv)

In the field of cross-modal retrieval, single encoder models tend to perform better than dual encoder models, but they suffer from high latency and low throughput. In this paper, we present a dual encoder model called BagFormer that utilizes a cross modal interaction mechanism to improve recall performance without sacrificing latency and throughput. BagFormer achieves this through the use of bag-wise interactions, which allow for the transformation of text to a more appropriate granularity and the incorporation of entity knowledge into the model. Our experiments demonstrate that BagFormer is able to achieve results comparable to state-of-the-art single encoder models in cross-modal retrieval tasks, while also offering efficient training and inference with 20.72 times lower latency and 25.74 times higher throughput.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 29 Dec 2022.

Builds on

27 of 33 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  12. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
15 more
  1. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  2. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  4. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  5. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  6. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  7. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  8. 2018Stacked Cross Attention for Image-Text Matchingcited by 414
  9. 2020IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrievalcited by 115
  10. 2022A Neural Corpus Indexer for Document Retrievalcited by 107
  11. 2020Graph Structured Network for Image-Text Matchingcited by 93
  12. 2021M6: A Chinese Multimodal Pretrainercited by 91
  13. 2017Automated Phrase Mining from Massive Text Corporacited by 70
  14. 2022ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrievalcited by 29
  15. 2019Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual Cross-modal Retrievalcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026EmbeddingRWKV: State-Centric Retrieval with Reusable Statescited by 0