mlsanityData to 7 Oct 2026

Modeling Context Between Objects for Referring Expression Understanding

arXiv 1608.00525 ↗First posted 1 Aug 2016cs.CVcited by 250

Not on the map. Why?

Cited by 30 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Referring expressions usually describe an object using properties of the object and relationships of the object with other objects. We propose a technique that integrates context between objects to understand referring expressions. Our approach uses an LSTM to learn the probability of a referring expression, with input features from a region and a context region. The context regions are discovered using multiple-instance learning (MIL) since annotations for context objects are generally not available for training. We utilize max-margin based MIL objective functions for training the LSTM. Experiments on the Google RefExp and UNC RefExp datasets show that modeling context between objects provides better performance than modeling only object properties. We also qualitatively show that our technique can ground a referring expression to its referred region along with the supporting context region.

Key facts. Cited by 250 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 30; 2023: 55; 2024: 49; 2025: 57; 2026: 36.

250 papers.

Sort:

By area:

  1. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  2. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  3. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  4. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  5. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  6. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  7. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  8. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  9. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  10. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  11. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  12. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
38 more
  1. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  2. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  3. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  4. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  5. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 109
  6. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  7. 2025MiMo-VL Technical Reportcited by 106
  8. 2021ArtEmis: Affective Language for Visual Artcited by 103
  9. 2023Contextual Object Detection with Multimodal Large Language Modelscited by 100
  10. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 100
  11. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  12. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 97
  13. 2023Referring Multi-Object Trackingcited by 91
  14. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  15. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferencescited by 76
  16. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  17. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  18. 2023CLIP as RNN: Segment Countless Visual Concepts without Training Endeavorcited by 69
  19. 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
  20. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 63
  21. 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
  22. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  23. 2023SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentationcited by 57
  24. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  25. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 53
  26. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Modelcited by 52
  27. 2024Toward Visual Grounding: A Surveycited by 52
  28. 2021Look Before You Leap: Learning Landmark Features for One-Stage Visual Groundingcited by 51
  29. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  30. 2022Towards Robust Referring Image Segmentationcited by 49
  31. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  32. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 43
  33. 2021Visual Grounding with Transformerscited by 42
  34. 2022TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformercited by 41
  35. 2023Mask Grounding for Referring Image Segmentationcited by 40
  36. 2024ReMamber: Referring Image Segmentation with Mamba Twistercited by 38
  37. 2022It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collectioncited by 36
  38. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36

50 most cited of 250 All 250 (JSON)