mlsanityData to 14 Oct 2026

Natural Language Object Retrieval

arXiv 1511.04164 ↗First posted 13 Nov 2015cs.CV cs.CLcited by 124

Not on the map. Why?

Abstract (arXiv)

In this paper, we address the task of natural language object retrieval, to localize a target object within a given image based on a natural language query of the object. Natural language object retrieval differs from text-based image retrieval task as it involves spatial information about objects within the scene and global scene context. To address this issue, we propose a novel Spatial Context Recurrent ConvNet (SCRC) model as scoring function on candidate boxes for object retrieval, integrating spatial configurations and global scene-level contextual information into the network. Our model processes query text, local image descriptors, spatial configurations and global context features through a recurrent network, outputs the probability of the query text conditioned on each candidate box as a score for the box, and can transfer visual-linguistic knowledge from image captioning domain to our task. Experimental results demonstrate that our method effectively utilizes both local and global information, outperforming previous baseline methods significantly on different datasets and scenarios, and can exploit large scale vision and language datasets for knowledge transfer.

Key facts. Cited by 124 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 30; 2022: 45; 2023: 26; 2024: 12; 2025: 6; 2026: 4.

124 papers.

Sort:

By area:

  1. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  2. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  3. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  4. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
  5. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  6. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  7. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 126
  8. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  9. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 123
  10. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  11. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  12. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
38 more
  1. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
  2. 2021Joint Learning of Localized Representations from Medical Images and Reportscited by 81
  3. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  4. 2022Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Groundingcited by 53
  5. 2021Look Before You Leap: Learning Landmark Features for One-Stage Visual Groundingcited by 52
  6. 2021D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Groundingcited by 52
  7. 2024Toward Visual Grounding: A Surveycited by 52
  8. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  9. 2022X-DETR: A Versatile Architecture for Instance-wise Vision-Language Taskscited by 43
  10. 2021Visual Grounding with Transformerscited by 42
  11. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  12. 2022TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformercited by 42
  13. 2023GREC: Generalized Referring Expression Comprehensioncited by 42
  14. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  15. 2022Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Groundingcited by 41
  16. 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
  17. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  18. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  19. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  20. 2022Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Groundingcited by 35
  21. 2023CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Groundingcited by 35
  22. 2021Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Imagescited by 33
  23. 2022Pseudo-Q: Generating Pseudo Language Queries for Visual Groundingcited by 33
  24. 2023GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Modelscited by 32
  25. 2024HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Groundingcited by 31
  26. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  27. 2022Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Groundingcited by 29
  28. 2021Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Networkcited by 28
  29. 2021The 5th AI City Challengecited by 28
  30. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  31. 2024OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modelingcited by 28
  32. 2021CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptionscited by 26
  33. 2022WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Trackingcited by 26
  34. 2022ComCLIP: Training-Free Compositional Image and Text Matchingcited by 26
  35. 2021Extending CLIP for Category-to-image Retrieval in E-commercecited by 24
  36. 2022Exploring Motion and Appearance Information for Temporal Sentence Groundingcited by 24
  37. 2022DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Groundingcited by 24
  38. 2021A Joint Network for Grasp Detection Conditioned on Natural Language Commandscited by 23

50 most cited of 124 All 124 (JSON)