# Interpretable and Globally Optimal Prediction for Textual Grounding using Image Concepts

arXiv: https://arxiv.org/abs/1803.11209 · HTML: https://mlsanity.com/paper/1803.11209/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2018.

## Built on by (most cited first, 15 of 15)

- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Look Before You Leap: Learning Landmark Features for One-Stage Visual Grounding](https://mlsanity.com/paper/2104.04386/) (2021)
- [Visual Grounding with Transformers](https://mlsanity.com/paper/2105.04281/) (2021)
- [Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations](https://mlsanity.com/paper/2206.15462/) (2022)
- [Adapting CLIP For Phrase Localization Without Further Training](https://mlsanity.com/paper/2204.03647/) (2022)
- [Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.05187/) (2021)
- [MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding](https://mlsanity.com/paper/2112.10728/) (2021)
- [Disentangled Motif-aware Graph Learning for Phrase Grounding](https://mlsanity.com/paper/2104.06008/) (2021)
- [Total Variation Optimization Layers for Computer Vision](https://mlsanity.com/paper/2204.03643/) (2022)
- [Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching](https://mlsanity.com/paper/2201.06686/) (2022)
- [Unified Open-Vocabulary Dense Visual Prediction](https://mlsanity.com/paper/2307.08238/) (2023)
- [A Better Loss for Visual-Textual Grounding](https://mlsanity.com/paper/2108.05308/) (2021)
- [Heatmap Regression without Soft-Argmax for Facial Landmark Detection](https://mlsanity.com/paper/2508.14929/) (2025)
- [Bridging the Gap Between Object Detection and User Intent via Query-Modulation](https://mlsanity.com/paper/2106.10258/) (2021)
- [3DWG: 3D Weakly Supervised Visual Grounding via Category and Instance-Level Alignment](https://mlsanity.com/paper/2505.01809/) (2025)

## Abstract (arXiv)

Textual grounding is an important but challenging task for human-computer interaction, robotics and knowledge mining. Existing algorithms generally formulate the task as selection from a set of bounding box proposals obtained from deep net based systems. In this work, we demonstrate that we can cast the problem of textual grounding into a unified framework that permits efficient search over all possible bounding boxes. Hence, the method is able to consider significantly more proposals and doesn't rely on a successful first stage hypothesizing bounding box proposals. Beyond, we demonstrate that the trained parameters of our model can be used as word-embeddings which capture spatial-image relationships and provide interpretability. Lastly, at the time of submission, our approach outperformed the current state-of-the-art methods on the Flickr 30k Entities and the ReferItGame dataset by 3.08% and 7.77% respectively.