mlsanityData to 7 Oct 2026

Stacked Cross Attention for Image-Text Matching

arXiv 1803.08024 ↗First posted 21 Mar 2018cs.CV cs.AI cs.LGcited by 414

Not on the map. Why?

Cited by 49 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we study the problem of image-text matching. Inferring the latent semantic alignment between objects or other salient stuff (e.g. snow, sky, lawn) and the corresponding words in sentences allows to capture fine-grained interplay between vision and language, and makes image-text matching more interpretable. Prior work either simply aggregates the similarity of all possible pairs of regions and words without attending differentially to more and less important words or regions, or uses a multi-step attentional process to capture limited number of semantic alignments which is less interpretable. In this paper, we present Stacked Cross Attention to discover the full latent alignments using both image regions and words in a sentence as context and infer image-text similarity. Our approach achieves the state-of-the-art results on the MS-COCO and Flickr30K datasets. On Flickr30K, our approach outperforms the current best methods by 22.1% relatively in text retrieval from image query, and 18.2% relatively in image retrieval with text query (based on Recall@1). On MS-COCO, our approach improves sentence retrieval by 17.8% relatively and image retrieval by 16.6% relatively (based on Recall@1 using the 5K test set). Code has been made available at: https://github.com/kuanghuei/SCAN.

Key facts. Cited by 414 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 85; 2022: 93; 2023: 88; 2024: 61; 2025: 56; 2026: 31.

414 papers.

Sort:

By area:

  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  2. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  4. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  5. 2021Transformers in Vision: A Surveycited by 1,009
  6. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  7. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 357
  8. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  9. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  10. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 190
  11. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  12. 2022A Survey of Vision-Language Pre-Trained Modelscited by 177
38 more
  1. 2022Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learningcited by 159
  2. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  3. 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 150
  4. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  5. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  6. 2021Similarity Reasoning and Filtration for Image-Text Matchingcited by 121
  7. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  8. 2021VinVL: Revisiting Visual Representations in Vision-Language Modelscited by 109
  9. 2021Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identificationcited by 108
  10. 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
  11. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
  12. 2021DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrievalcited by 102
  13. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trendscited by 93
  14. 2021Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Searchcited by 90
  15. 2021HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrievalcited by 87
  16. 2021E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learningcited by 84
  17. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
  18. 2022Parameter-efficient Model Adaptation for Vision Transformerscited by 81
  19. 2022CLIP-Driven Fine-grained Text-Image Person Re-identificationcited by 79
  20. 2023Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluationcited by 78
  21. 2022Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrievalcited by 77
  22. 2023Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmarkcited by 75
  23. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  24. 2022HiTeA: Hierarchical Temporal-Aware Video-Language Pre-trainingcited by 72
  25. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  26. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  27. 2021Kaleido-BERT: Vision-Language Pre-training on Fashion Domaincited by 67
  28. 2021Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-trainingcited by 67
  29. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  30. 2022See Finer, See More: Implicit Modality Alignment for Text-based Person Retrievalcited by 65
  31. 2022Learning Granularity-Unified Representations for Text-to-Image Person Re-identificationcited by 62
  32. 2021Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Searchcited by 60
  33. 2021TIPCB: A Simple but Effective Part-based Convolutional Baseline for Text-based Person Searchcited by 57
  34. 2021Object-aware Video-language Pre-training for Retrievalcited by 57
  35. 2024Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attentioncited by 57
  36. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  37. 2022Image-text Retrieval: A Survey on Recent Research and Developmentcited by 55
  38. 2022Disentangling Visual Embeddings for Attributes and Objectscited by 55

50 most cited of 414 All 414 (JSON)