# Learning Cross-modal Context Graph for Visual Grounding

arXiv: https://arxiv.org/abs/1911.09042 · HTML: https://mlsanity.com/paper/1911.09042/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2019.

## Built on by (most cited first, 30 of 49)

- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring](https://mlsanity.com/paper/2103.01128/) (2021)
- [EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Grounding](https://mlsanity.com/paper/2209.14941/) (2022)
- [LanguageRefer: Spatial-Language Model for 3D Visual Grounding](https://mlsanity.com/paper/2107.03438/) (2021)
- [Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloud](https://mlsanity.com/paper/2103.16381/) (2021)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding](https://mlsanity.com/paper/2203.15442/) (2022)
- [Toward Visual Grounding: A Survey](https://mlsanity.com/paper/2412.20206/) (2024)
- [TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer](https://mlsanity.com/paper/2206.06619/) (2022)
- [Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Grounding](https://mlsanity.com/paper/2211.14241/) (2022)
- [Relation-aware Instance Refinement for Weakly Supervised Visual Grounding](https://mlsanity.com/paper/2103.12989/) (2021)
- [Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding](https://mlsanity.com/paper/2209.13959/) (2022)
- [DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding](https://mlsanity.com/paper/2211.15516/) (2022)
- [KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation](https://mlsanity.com/paper/2109.10504/) (2021)
- [Vision+X: A Survey on Multimodal Learning in the Light of Data](https://mlsanity.com/paper/2210.02884/) (2022)
- [Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructions](https://mlsanity.com/paper/2205.04028/) (2022)
- [Disentangled Motif-aware Graph Learning for Phrase Grounding](https://mlsanity.com/paper/2104.06008/) (2021)
- [Word2Pix: Word to Pixel Cross Attention Transformer in Visual Grounding](https://mlsanity.com/paper/2108.00205/) (2021)
- [Deconfounded Visual Grounding](https://mlsanity.com/paper/2112.15324/) (2021)
- [Interpretable Multimodal Misinformation Detection with Logic Reasoning](https://mlsanity.com/paper/2305.05964/) (2023)
- [PPMN: Pixel-Phrase Matching Network for One-Stage Panoptic Narrative Grounding](https://mlsanity.com/paper/2208.05647/) (2022)
- [X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2107.11576/) (2021)
- [AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relations](https://mlsanity.com/paper/2504.07836/) (2025)
- [Comprehensive Multi-Modal Interactions for Referring Image Segmentation](https://mlsanity.com/paper/2104.10412/) (2021)
- [Grounded Image Text Matching with Mismatched Relation Reasoning](https://mlsanity.com/paper/2308.01236/) (2023)
- [Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching](https://mlsanity.com/paper/2201.06686/) (2022)
- [OV-VG: A Benchmark for Open-Vocabulary Visual Grounding](https://mlsanity.com/paper/2310.14374/) (2023)
- [Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?](https://mlsanity.com/paper/2106.04232/) (2021)
- [Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding](https://mlsanity.com/paper/2203.05186/) (2022)
- [Cross-Modal Alignment Learning of Vision-Language Conceptual Systems](https://mlsanity.com/paper/2208.01744/) (2022)

## Abstract (arXiv)

Visual grounding is a ubiquitous building block in many vision-language tasks and yet remains challenging due to large variations in visual and linguistic features of grounding entities, strong context effect and the resulting semantic ambiguities. Prior works typically focus on learning representations of individual phrases with limited context information. To address their limitations, this paper proposes a language-guided graph representation to capture the global context of grounding entities and their relations, and develop a cross-modal graph matching strategy for the multiple-phrase visual grounding task. In particular, we introduce a modular graph neural network to compute context-aware representations of phrases and object proposals respectively via message propagation, followed by a graph-based matching module to generate globally consistent localization of grounding phrases. We train the entire graph neural network jointly in a two-stage strategy and evaluate it on the Flickr30K Entities benchmark. Extensive experiments show that our method outperforms the prior state of the arts by a sizable margin, evidencing the efficacy of our grounding framework. Code is available at "https://github.com/youngfly11/LCMCG-PyTorch".