mlsanityData to 7 Oct 2026

Modeling Context in Referring Expressions

arXiv 1608.00272 ↗First posted 31 Jul 2016cs.CV cs.CLcited by 1,054

Not on the map. Why?

Cited by 355 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Humans refer to objects in their environments all the time, especially in dialogue with other people. We explore generating and comprehending natural language referring expressions for objects in images. In particular, we focus on incorporating better measures of visual context into referring expression models and find that visual comparison to other objects within an image helps improve performance significantly. We also develop methods to tie the language generation process together, so that we generate expressions for all objects of a particular category jointly. Evaluation on three recent datasets - RefCOCO, RefCOCO+, and RefCOCOg, shows the advantages of our methods for both referring expression generation and comprehension.

Key facts. Cited by 1,054 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 61; 2022: 112; 2023: 208; 2024: 225; 2025: 232; 2026: 215.

1,054 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  5. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  6. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  7. 2021Transformers in Vision: A Surveycited by 1,009
  8. 2023A Survey on Multimodal Large Language Modelscited by 898
  9. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  10. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  11. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  12. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
38 more
  1. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  2. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  3. 2023Segment Everything Everywhere All at Oncecited by 546
  4. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  5. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  6. 2021Image Segmentation Using Text and Image Promptscited by 509
  7. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  8. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  9. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  10. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  11. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  12. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  13. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  14. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  15. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  16. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  17. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  18. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 320
  19. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  20. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  21. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 300
  22. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  23. 2022Visual Spatial Reasoningcited by 288
  24. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  25. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  26. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  27. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  28. 2021Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Conceptscited by 255
  29. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  30. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
  31. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  32. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 234
  33. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  34. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  35. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
  36. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  37. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  38. 2023Tracking Anything with Decoupled Video Segmentationcited by 197

50 most cited of 1,054 All 1,054 (JSON)