# Discriminative Triad Matching and Reconstruction for Weakly Referring Expression Grounding

arXiv: https://arxiv.org/abs/2106.04053 · HTML: https://mlsanity.com/paper/2106.04053/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [Cross-task weakly supervised learning from instructional videos](https://mlsanity.com/paper/1903.08225/) (2019)
- [FVQA: Fact-based Visual Question Answering](https://mlsanity.com/paper/1606.05433/) (2016)
- [Grounding of Textual Phrases in Images by Reconstruction](https://mlsanity.com/paper/1511.03745/) (2015)
- [Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networks](https://mlsanity.com/paper/1812.04794/) (2018)
- [Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model](https://mlsanity.com/paper/1912.09637/) (2019)
- [Learning to Compose and Reason with Language Tree Structures for Visual Grounding](https://mlsanity.com/paper/1906.01784/) (2019)
- [Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing](https://mlsanity.com/paper/1903.00839/) (2019)
- [Normalized and Geometry-Aware Self-Attention Network for Image Captioning](https://mlsanity.com/paper/2003.08897/) (2020)
- [Image Captioning and Visual Question Answering Based on Attributes and External Knowledge](https://mlsanity.com/paper/1603.02814/) (2016)
- [Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding](https://mlsanity.com/paper/1908.10568/) (2019)
- [Knowledge Aided Consistency for Weakly Supervised Phrase Grounding](https://mlsanity.com/paper/1803.03879/) (2018)
- [Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Grounding](https://mlsanity.com/paper/1909.02860/) (2019)
- [Exploring Overall Contextual Information for Image Captioning in Human-Like Cognitive Style](https://mlsanity.com/paper/1910.06475/) (2019)

## Built on by (most cited first, 21 of 21)

- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension](https://mlsanity.com/paper/2204.05991/) (2022)
- [Toward Visual Grounding: A Survey](https://mlsanity.com/paper/2412.20206/) (2024)
- [CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding](https://mlsanity.com/paper/2305.08685/) (2023)
- [Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding](https://mlsanity.com/paper/2203.08481/) (2022)
- [Vision+X: A Survey on Multimodal Learning in the Light of Data](https://mlsanity.com/paper/2210.02884/) (2022)
- [A Weakly Supervised Learning Framework for Salient Object Detection via Hybrid Labels](https://mlsanity.com/paper/2209.02957/) (2022)
- [GroundVLP: Harnessing Zero-shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object Detection](https://mlsanity.com/paper/2312.15043/) (2023)
- [Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment](https://mlsanity.com/paper/2208.13628/) (2022)
- [Democracy Does Matter: Comprehensive Feature Mining for Co-Salient Object Detection](https://mlsanity.com/paper/2203.05787/) (2022)
- [Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learning](https://mlsanity.com/paper/2212.10278/) (2022)
- [Weakly-Supervised Video Object Grounding via Causal Intervention](https://mlsanity.com/paper/2112.00475/) (2021)
- [WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2505.18686/) (2025)
- [Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding](https://mlsanity.com/paper/2203.05186/) (2022)
- [Seeing the Trees for the Forest: Rethinking Weakly-Supervised Medical Visual Grounding](https://mlsanity.com/paper/2505.15123/) (2025)
- [AlignCAT: Visual-Linguistic Alignment of Category and Attribute for Weakly Supervised Visual Grounding](https://mlsanity.com/paper/2508.03201/) (2025)
- [LIHE: Linguistic Instance-Split Hyperbolic-Euclidean Framework for Generalized Weakly-Supervised Referring Expression Comprehension](https://mlsanity.com/paper/2511.12020/) (2025)
- [TALENT: Target-aware Efficient Tuning for Referring Image Segmentation](https://mlsanity.com/paper/2604.00609/) (2026)
- [Differentiated Relevances Embedding for Group-based Referring Expression Comprehension](https://mlsanity.com/paper/2203.06382/) (2022)
- [Training-free Clothing Region of Interest Self-correction for Virtual Try-On](https://mlsanity.com/paper/2512.07126/) (2025)
- [Learning Structured Visual Compositional Representations for Weakly Supervised Referring Expression Comprehension](https://mlsanity.com/paper/2607.04638/) (2026)

## Abstract (arXiv)

In this paper, we are tackling the weakly-supervised referring expression grounding task, for the localization of a referent object in an image according to a query sentence, where the mapping between image regions and queries are not available during the training stage. In traditional methods, an object region that best matches the referring expression is picked out, and then the query sentence is reconstructed from the selected region, where the reconstruction difference serves as the loss for back-propagation. The existing methods, however, conduct both the matching and the reconstruction approximately as they ignore the fact that the matching correctness is unknown. To overcome this limitation, a discriminative triad is designed here as the basis to the solution, through which a query can be converted into one or multiple discriminative triads in a very scalable way. Based on the discriminative triad, we further propose the triad-level matching and reconstruction modules which are lightweight yet effective for the weakly-supervised training, making it three times lighter and faster than the previous state-of-the-art methods. One important merit of our work is its superior performance despite the simple and neat design. Specifically, the proposed method achieves a new state-of-the-art accuracy when evaluated on RefCOCO (39.21%), RefCOCO+ (39.18%) and RefCOCOg (43.24%) datasets, that is 4.17%, 4.08% and 7.8% higher than the previous one, respectively.