# Learning Dual Semantic Relations with Graph Attention for Image-Text Matching

arXiv: https://arxiv.org/abs/2010.11550 · HTML: https://mlsanity.com/paper/2010.11550/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 30 of 30)

- [Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrieval](https://mlsanity.com/paper/2210.08908/) (2022)
- [Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learning](https://mlsanity.com/paper/2212.10278/) (2022)
- [CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval](https://mlsanity.com/paper/2208.09843/) (2022)
- [USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval](https://mlsanity.com/paper/2301.06844/) (2023)
- [Is CLIP the main roadblock for fine-grained open-world perception?](https://mlsanity.com/paper/2404.03539/) (2024)
- [Composing Object Relations and Attributes for Image-Text Matching](https://mlsanity.com/paper/2406.11820/) (2024)
- [Efficient Token-Guided Image-Text Retrieval with Consistent Multimodal Contrastive Training](https://mlsanity.com/paper/2306.08789/) (2023)
- [HGAN: Hierarchical Graph Alignment Network for Image-Text Retrieval](https://mlsanity.com/paper/2212.08281/) (2022)
- [ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval](https://mlsanity.com/paper/2207.14757/) (2022)
- [Vision-Language Matching for Text-to-Image Synthesis via Generative Adversarial Networks](https://mlsanity.com/paper/2208.09596/) (2022)
- [Hierarchical Matching and Reasoning for Multi-Query Image Retrieval](https://mlsanity.com/paper/2306.14460/) (2023)
- [Two-stream Hierarchical Similarity Reasoning for Image-text Matching](https://mlsanity.com/paper/2203.05349/) (2022)
- [Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching](https://mlsanity.com/paper/2303.00181/) (2023)
- [RISAM: Referring Image Segmentation via Mutual-Aware Attention Features](https://mlsanity.com/paper/2311.15727/) (2023)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [Referring Remote Sensing Image Segmentation with Cross-view Semantics Interaction Network](https://mlsanity.com/paper/2508.01331/) (2025)
- [A Unified Two-Stage Group Semantics Propagation and Contrastive Learning Network for Co-Saliency Detection](https://mlsanity.com/paper/2208.06615/) (2022)
- [MALM: Mask Augmentation based Local Matching for Food-Recipe Retrieval](https://mlsanity.com/paper/2305.11327/) (2023)
- [Text-Guided Face Recognition using Multi-Granularity Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2312.09367/) (2023)
- [Spatiotemporal Graph Guided Multi-modal Network for Livestreaming Product Retrieval](https://mlsanity.com/paper/2407.16248/) (2024)
- [Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval](https://mlsanity.com/paper/2506.21538/) (2025)
- [Ambiguity-Aware and High-Order Relation Learning for Multi-Grained Image-Text Matching](https://mlsanity.com/paper/2507.09256/) (2025)
- [Reliable Cross-modal Alignment via Prototype Iterative Construction](https://mlsanity.com/paper/2510.11175/) (2025)
- [Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval](https://mlsanity.com/paper/2603.26787/) (2026)
- [Contrastive Cross-Modal Knowledge Sharing Pre-training for Vision-Language Representation Learning and Retrieval](https://mlsanity.com/paper/2207.00733/) (2022)
- [Efficient Image-Text Retrieval via Keyword-Guided Pre-Screening](https://mlsanity.com/paper/2303.07740/) (2023)
- [Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval](https://mlsanity.com/paper/2305.16566/) (2023)
- [Hire: Hybrid-modal Interaction with Multiple Relational Enhancements for Image-Text Matching](https://mlsanity.com/paper/2406.18579/) (2024)
- [OS-HGAdapter: Open Semantic Hypergraph Adapter for Large Language Models Assisted Entropy-Enhanced Image-Text Alignment](https://mlsanity.com/paper/2510.13131/) (2025)
- [Aligning the True Semantics: Constrained Decoupling and Distribution Sampling for Cross-Modal Alignment](https://mlsanity.com/paper/2603.05566/) (2026)

## Abstract (arXiv)

Image-Text Matching is one major task in cross-modal information processing. The main challenge is to learn the unified visual and textual representations. Previous methods that perform well on this task primarily focus on not only the alignment between region features in images and the corresponding words in sentences, but also the alignment between relations of regions and relational words. However, the lack of joint learning of regional features and global features will cause the regional features to lose contact with the global context, leading to the mismatch with those non-object words which have global meanings in some sentences. In this work, in order to alleviate this issue, it is necessary to enhance the relations between regions and the relations between regional and global concepts to obtain a more accurate visual representation so as to be better correlated to the corresponding text. Thus, a novel multi-level semantic relations enhancement approach named Dual Semantic Relations Attention Network(DSRAN) is proposed which mainly consists of two modules, separate semantic relations module and the joint semantic relations module. DSRAN performs graph attention in both modules respectively for region-level relations enhancement and regional-global relations enhancement at the same time. With these two modules, different hierarchies of semantic relations are learned simultaneously, thus promoting the image-text matching process by providing more information for the final visual representation. Quantitative experimental results have been performed on MS-COCO and Flickr30K and our method outperforms previous approaches by a large margin due to the effectiveness of the dual semantic relations learning scheme. Codes are available at https://github.com/kywen1119/DSRAN.