# Look Before You Leap: Learning Landmark Features for One-Stage Visual Grounding

arXiv: https://arxiv.org/abs/2104.04386 · HTML: https://mlsanity.com/paper/2104.04386/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 49 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [Libra R-CNN: Towards Balanced Learning for Object Detection](https://mlsanity.com/paper/1904.02701/) (2019)
- [Modeling Context Between Objects for Referring Expression Understanding](https://mlsanity.com/paper/1608.00525/) (2016)
- [A Fast and Accurate One-Stage Approach to Visual Grounding](https://mlsanity.com/paper/1908.06354/) (2019)
- [Learning to Assemble Neural Module Tree Networks for Visual Grounding](https://mlsanity.com/paper/1812.03299/) (2018)
- [Improving One-stage Visual Grounding by Recursive Sub-query Construction](https://mlsanity.com/paper/2008.01059/) (2020)
- [Natural Language Object Retrieval](https://mlsanity.com/paper/1511.04164/) (2015)
- [Learning Two-Branch Neural Networks for Image-Text Matching Tasks](https://mlsanity.com/paper/1704.03470/) (2017)
- [A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehension](https://mlsanity.com/paper/1909.07072/) (2019)

## Built on by (most cited first, 30 of 52)

- [EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain](https://mlsanity.com/paper/2401.16822/) (2024)
- [SeqTR: A Simple yet Universal Network for Visual Grounding](https://mlsanity.com/paper/2203.16265/) (2022)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoning](https://mlsanity.com/paper/2205.00272/) (2022)
- [Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding](https://mlsanity.com/paper/2203.15442/) (2022)
- [Toward Visual Grounding: A Survey](https://mlsanity.com/paper/2412.20206/) (2024)
- [RemoteSAM: Towards Segment Anything for Earth Observation](https://mlsanity.com/paper/2505.18022/) (2025)
- [SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusion](https://mlsanity.com/paper/2409.17531/) (2024)
- [Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding](https://mlsanity.com/paper/2209.13959/) (2022)
- [Parallel Vertex Diffusion for Unified Visual Grounding](https://mlsanity.com/paper/2303.07216/) (2023)
- [Unifying Visual and Vision-Language Tracking via Contrastive Learning](https://mlsanity.com/paper/2401.11228/) (2024)
- [Shatter and Gather: Learning Referring Image Segmentation with Text Supervision](https://mlsanity.com/paper/2308.15512/) (2023)
- [Context Disentangling and Prototype Inheriting for Robust Visual Grounding](https://mlsanity.com/paper/2312.11967/) (2023)
- [Mono3DVG: 3D Visual Grounding in Monocular Images](https://mlsanity.com/paper/2312.08022/) (2023)
- [An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Grounding](https://mlsanity.com/paper/2408.01120/) (2024)
- [Referring Camouflaged Object Detection](https://mlsanity.com/paper/2306.07532/) (2023)
- [Towards Language-guided Visual Recognition via Dynamic Convolutions](https://mlsanity.com/paper/2110.08797/) (2021)
- [Advancing Visual Grounding with Scene Knowledge: Benchmark and Method](https://mlsanity.com/paper/2307.11558/) (2023)
- [SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation](https://mlsanity.com/paper/2406.01451/) (2024)
- [Deconfounded Visual Grounding](https://mlsanity.com/paper/2112.15324/) (2021)
- [SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding](https://mlsanity.com/paper/2407.03200/) (2024)
- [Correspondence Matters for Video Referring Expression Comprehension](https://mlsanity.com/paper/2207.10400/) (2022)
- [WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2505.18686/) (2025)
- [Grounded Image Text Matching with Mismatched Relation Reasoning](https://mlsanity.com/paper/2308.01236/) (2023)
- [Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching](https://mlsanity.com/paper/2201.06686/) (2022)
- [Visual Grounding with Attention-Driven Constraint Balancing](https://mlsanity.com/paper/2407.03243/) (2024)
- [Learning Point-Language Hierarchical Alignment for 3D Visual Grounding](https://mlsanity.com/paper/2210.12513/) (2022)

## Abstract (arXiv)

An LBYL (`Look Before You Leap') Network is proposed for end-to-end trainable one-stage visual grounding. The idea behind LBYL-Net is intuitive and straightforward: we follow a language's description to localize the target object based on its relative spatial relation to `Landmarks', which is characterized by some spatial positional words and some descriptive words about the object. The core of our LBYL-Net is a landmark feature convolution module that transmits the visual features with the guidance of linguistic description along with different directions. Consequently, such a module encodes the relative spatial positional relations between the current object and its context. Then we combine the contextual information from the landmark feature convolution module with the target's visual features for grounding. To make this landmark feature convolution light-weight, we introduce a dynamic programming algorithm (termed dynamic max pooling) with low complexity to extract the landmark feature. Thanks to the landmark feature convolution module, we mimic the human behavior of `Look Before You Leap' to design an LBYL-Net, which takes full consideration of contextual information. Extensive experiments show our method's effectiveness in four grounding datasets. Specifically, our LBYL-Net outperforms all state-of-the-art two-stage and one-stage methods on ReferitGame. On RefCOCO and RefCOCO+, Our LBYL-Net also achieves comparable results or even better results than existing one-stage methods.