# Proposal-free One-stage Referring Expression via Grid-Word Cross-Attention

arXiv: https://arxiv.org/abs/2105.02061 · HTML: https://mlsanity.com/paper/2105.02061/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [A Fast and Accurate One-Stage Approach to Visual Grounding](https://mlsanity.com/paper/1908.06354/) (2019)
- [Improving One-stage Visual Grounding by Recursive Sub-query Construction](https://mlsanity.com/paper/2008.01059/) (2020)
- [A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehension](https://mlsanity.com/paper/1909.07072/) (2019)
- [Dynamic Graph Attention for Referring Expression Comprehension](https://mlsanity.com/paper/1909.08164/) (2019)
- [Modeling Relationships in Referential Expressions with Compositional Modular Networks](https://mlsanity.com/paper/1611.09978/) (2016)
- [Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networks](https://mlsanity.com/paper/1812.04794/) (2018)
- [Grounding Referring Expressions in Images by Variational Context](https://mlsanity.com/paper/1712.01892/) (2017)
- [A Joint Speaker-Listener-Reinforcer Model for Referring Expressions](https://mlsanity.com/paper/1612.09542/) (2016)
- [Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queries](https://mlsanity.com/paper/1711.06370/) (2017)
- [Real-Time Referring Expression Comprehension by Single-Stage Grounding Network](https://mlsanity.com/paper/1812.03426/) (2018)
- [VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation](https://mlsanity.com/paper/1708.04686/) (2017)
- [AMC: Attention guided Multi-modal Correlation Learning for Image Search](https://mlsanity.com/paper/1704.00763/) (2017)

## Built on by (most cited first, 5 of 5)

- [Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey](https://mlsanity.com/paper/2302.10035/) (2023)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Multi-Attention Network for Compressed Video Referring Object Segmentation](https://mlsanity.com/paper/2207.12622/) (2022)
- [Deconfounded Visual Grounding](https://mlsanity.com/paper/2112.15324/) (2021)
- [One for All: One-stage Referring Expression Comprehension with Dynamic Reasoning](https://mlsanity.com/paper/2208.00361/) (2022)

## Abstract (arXiv)

Referring Expression Comprehension (REC) has become one of the most important tasks in visual reasoning, since it is an essential step for many vision-and-language tasks such as visual question answering. However, it has not been widely used in many downstream tasks because it suffers 1) two-stage methods exist heavy computation cost and inevitable error accumulation, and 2) one-stage methods have to depend on lots of hyper-parameters (such as anchors) to generate bounding box. In this paper, we present a proposal-free one-stage (PFOS) model that is able to regress the region-of-interest from the image, based on a textual query, in an end-to-end manner. Instead of using the dominant anchor proposal fashion, we directly take the dense-grid of an image as input for a cross-attention transformer that learns grid-word correspondences. The final bounding box is predicted directly from the image without the time-consuming anchor selection process that previous methods suffer. Our model achieves the state-of-the-art performance on four referring expression datasets with higher efficiency, comparing to previous best one-stage and two-stage methods.