mlsanityData to 14 Oct 2026

Generation and Comprehension of Unambiguous Object Descriptions

arXiv 1511.02283 ↗First posted 7 Nov 2015cs.CV cs.CL cs.LGcited by 904

Not on the map. Why?

Cited by 268 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a method that can generate an unambiguous description (known as a referring expression) of a specific object or region in an image, and which can also comprehend or interpret such an expression to infer which object is being described. We show that our method outperforms previous methods that generate descriptions of objects without taking into account other potentially ambiguous objects in the scene. Our model is inspired by recent successes of deep learning methods for image captioning, but while image captioning is difficult to evaluate, our task allows for easy objective evaluation. We also present a new large-scale dataset for referring expressions, based on MS-COCO. We have released the dataset and a toolbox for visualization and evaluation, see https://github.com/mjhucla/Google_Refexp_toolbox

Key facts. Cited by 904 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 68; 2022: 93; 2023: 182; 2024: 212; 2025: 188; 2026: 160.

904 papers.

Sort:

By area:

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  7. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  10. 2021Transformers in Vision: A Surveycited by 1,013
  11. 2023A Survey on Multimodal Large Language Modelscited by 909
  12. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
38 more
  1. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  2. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  3. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  4. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  5. 2025Qwen2.5-Omni Technical Reportcited by 736
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  7. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  8. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  9. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  10. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  11. 2023Segment Everything Everywhere All at Oncecited by 548
  12. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  13. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  14. 2021Image Segmentation Using Text and Image Promptscited by 512
  15. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  16. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  17. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  18. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  19. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  20. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  21. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  22. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  23. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  24. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  25. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  26. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  27. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  28. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  29. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  30. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  31. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  32. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  33. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  34. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  35. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  36. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  37. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
  38. 2023Tracking Anything with Decoupled Video Segmentationcited by 197

50 most cited of 904 All 904 (JSON)