# Graph Structured Network for Image-Text Matching

arXiv: https://arxiv.org/abs/2004.00277 · HTML: https://mlsanity.com/paper/2004.00277/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2020.

## Built on by (most cited first, 30 of 93)

- [InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring](https://mlsanity.com/paper/2103.01128/) (2021)
- [Vision-Language Pre-training: Basics, Recent Advances, and Future Trends](https://mlsanity.com/paper/2210.09263/) (2022)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding](https://mlsanity.com/paper/2108.02388/) (2021)
- [Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering](https://mlsanity.com/paper/2104.14085/) (2021)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Multimodal learning with graphs](https://mlsanity.com/paper/2209.03299/) (2022)
- [Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search](https://mlsanity.com/paper/2208.14365/) (2022)
- [ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval](https://mlsanity.com/paper/2203.16778/) (2022)
- [Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancement](https://mlsanity.com/paper/2210.03501/) (2022)
- [Noisy Correspondence Learning with Meta Similarity Correction](https://mlsanity.com/paper/2304.06275/) (2023)
- [Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrieval](https://mlsanity.com/paper/2108.02417/) (2021)
- [Cross-modal Active Complementary Learning with Self-refining Correspondence](https://mlsanity.com/paper/2310.17468/) (2023)
- [Learning Relation Alignment for Calibrated Cross-modal Retrieval](https://mlsanity.com/paper/2105.13868/) (2021)
- [Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.05187/) (2021)
- [Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrieval](https://mlsanity.com/paper/2203.03853/) (2022)
- [Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructions](https://mlsanity.com/paper/2205.04028/) (2022)
- [Unified Pre-training with Pseudo Texts for Text-To-Image Person Re-identification](https://mlsanity.com/paper/2309.01420/) (2023)
- [Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis](https://mlsanity.com/paper/2208.10970/) (2022)
- [Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translation](https://mlsanity.com/paper/2402.17144/) (2024)
- [Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrieval](https://mlsanity.com/paper/2210.08908/) (2022)
- [Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching](https://mlsanity.com/paper/2110.02623/) (2021)
- [CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval](https://mlsanity.com/paper/2208.09843/) (2022)
- [USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval](https://mlsanity.com/paper/2301.06844/) (2023)
- [OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graph](https://mlsanity.com/paper/2602.05576/) (2026)
- [Composing Object Relations and Attributes for Image-Text Matching](https://mlsanity.com/paper/2406.11820/) (2024)
- [MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning](https://mlsanity.com/paper/2601.22416/) (2026)
- [Plug-and-Play Regulators for Image-Text Matching](https://mlsanity.com/paper/2303.13371/) (2023)
- [Rethinking Benchmarks for Cross-modal Image-text Retrieval](https://mlsanity.com/paper/2304.10824/) (2023)
- [Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination](https://mlsanity.com/paper/2308.04380/) (2023)

## Abstract (arXiv)

Image-text matching has received growing interest since it bridges vision and language. The key challenge lies in how to learn correspondence between image and text. Existing works learn coarse correspondence based on object co-occurrence statistics, while failing to learn fine-grained phrase correspondence. In this paper, we present a novel Graph Structured Matching Network (GSMN) to learn fine-grained correspondence. The GSMN explicitly models object, relation and attribute as a structured phrase, which not only allows to learn correspondence of object, relation and attribute separately, but also benefits to learn fine-grained correspondence of structured phrase. This is achieved by node-level matching and structure-level matching. The node-level matching associates each node with its relevant nodes from another modality, where the node can be object, relation or attribute. The associated nodes then jointly infer fine-grained correspondence by fusing neighborhood associations at structure-level matching. Comprehensive experiments show that GSMN outperforms state-of-the-art methods on benchmarks, with relative Recall@1 improvements of nearly 7% and 2% on Flickr30K and MSCOCO, respectively. Code will be released at: https://github.com/CrossmodalGroup/GSMN.