# Query-guided Regression Network with Context Policy for Phrase Grounding

arXiv: https://arxiv.org/abs/1708.01676 · HTML: https://mlsanity.com/paper/1708.01676/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2017.

## Built on by (most cited first, 30 of 34)

- [TransVG: End-to-End Visual Grounding with Transformers](https://mlsanity.com/paper/2104.08541/) (2021)
- [Open-Vocabulary DETR with Conditional Matching](https://mlsanity.com/paper/2203.11876/) (2022)
- [Referring Transformer: A One-step Approach to Multi-task Visual Grounding](https://mlsanity.com/paper/2106.03089/) (2021)
- [DRAMA: Joint Risk Localization and Captioning in Driving](https://mlsanity.com/paper/2209.10767/) (2022)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding](https://mlsanity.com/paper/2108.02388/) (2021)
- [Toward Visual Grounding: A Survey](https://mlsanity.com/paper/2412.20206/) (2024)
- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [Visual Grounding with Transformers](https://mlsanity.com/paper/2105.04281/) (2021)
- [TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer](https://mlsanity.com/paper/2206.06619/) (2022)
- [Relation-aware Instance Refinement for Weakly Supervised Visual Grounding](https://mlsanity.com/paper/2103.12989/) (2021)
- [Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Images](https://mlsanity.com/paper/2103.07894/) (2021)
- [Entity-enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding](https://mlsanity.com/paper/2207.08386/) (2022)
- [Mono3DVG: 3D Visual Grounding in Monocular Images](https://mlsanity.com/paper/2312.08022/) (2023)
- [Adapting CLIP For Phrase Localization Without Further Training](https://mlsanity.com/paper/2204.03647/) (2022)
- [Skimming, Locating, then Perusing: A Human-Like Framework for Natural Language Video Localization](https://mlsanity.com/paper/2207.13450/) (2022)
- [Disentangled Motif-aware Graph Learning for Phrase Grounding](https://mlsanity.com/paper/2104.06008/) (2021)
- [SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation](https://mlsanity.com/paper/2406.01451/) (2024)
- [Detector-Free Weakly Supervised Grounding by Separation](https://mlsanity.com/paper/2104.09829/) (2021)
- [Deconfounded Visual Grounding](https://mlsanity.com/paper/2112.15324/) (2021)
- [What is Where by Looking: Weakly-Supervised Open-World Phrase-Grounding without Text Inputs](https://mlsanity.com/paper/2206.09358/) (2022)
- [MUG: Interactive Multimodal Grounding on User Interfaces](https://mlsanity.com/paper/2209.15099/) (2022)
- [TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System](https://mlsanity.com/paper/2311.06622/) (2023)
- [Who are you referring to? Coreference resolution in image narrations](https://mlsanity.com/paper/2211.14563/) (2022)
- [DATE: Domain Adaptive Product Seeker for E-commerce](https://mlsanity.com/paper/2304.03669/) (2023)
- [Box-based Refinement for Weakly Supervised and Unsupervised Localization Tasks](https://mlsanity.com/paper/2309.03874/) (2023)
- [Few-Shot Visual Grounding for Natural Human-Robot Interaction](https://mlsanity.com/paper/2103.09720/) (2021)
- [VL-NMS: Breaking Proposal Bottlenecks in Two-Stage Visual-Language Matching](https://mlsanity.com/paper/2105.05636/) (2021)
- [A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models](https://mlsanity.com/paper/2309.02691/) (2023)
- [A Better Loss for Visual-Textual Grounding](https://mlsanity.com/paper/2108.05308/) (2021)

## Abstract (arXiv)

Given a textual description of an image, phrase grounding localizes objects in the image referred by query phrases in the description. State-of-the-art methods address the problem by ranking a set of proposals based on the relevance to each query, which are limited by the performance of independent proposal generation systems and ignore useful cues from context in the description. In this paper, we adopt a spatial regression method to break the performance limit, and introduce reinforcement learning techniques to further leverage semantic context information. We propose a novel Query-guided Regression network with Context policy (QRC Net) which jointly learns a Proposal Generation Network (PGN), a Query-guided Regression Network (QRN) and a Context Policy Network (CPN). Experiments show QRC Net provides a significant improvement in accuracy on two popular datasets: Flickr30K Entities and Referit Game, with 14.25% and 17.14% increase over the state-of-the-arts respectively.