mlsanityData to 14 Oct 2026

Whether you can locate or not? Interactive Referring Expression Generation

arXiv 2308.09977 ↗First posted 19 Aug 2023cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Referring Expression Generation (REG) aims to generate unambiguous Referring Expressions (REs) for objects in a visual scene, with a dual task of Referring Expression Comprehension (REC) to locate the referred object. Existing methods construct REG models independently by using only the REs as ground truth for model training, without considering the potential interaction between REG and REC models. In this paper, we propose an Interactive REG (IREG) model that can interact with a real REC model, utilizing signals indicating whether the object is located and the visual region located by the REC model to gradually modify REs. Our experimental results on three RE benchmark datasets, RefCOCO, RefCOCO+, and RefCOCOg show that IREG outperforms previous state-of-the-art methods on popular evaluation metrics. Furthermore, a human evaluation shows that IREG generates better REs with the capability of interaction.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 19 Aug 2023.

Builds on

21 of 32 references.

References · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  6. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  7. 2016Modeling Context in Referring Expressionscited by 1,063
  8. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  9. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  10. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  11. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  12. 2020UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learningcited by 239
9 more
  1. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  2. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  3. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 140
  4. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 138
  5. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  6. 2016A Joint Speaker-Listener-Reinforcer Model for Referring Expressionscited by 74
  7. 2020Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Groundingcited by 50
  8. 2021Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learningcited by 20
  9. 2022Towards Unifying Reference Expression Generation and Comprehensioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perceptioncited by 39
  2. 2026GREx: Generalized Referring Expression Segmentation, Comprehension, and Generationcited by 6
  3. 2024Multimodal Transformer for Comics Text-Clozecited by 5
  4. 2024Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Groundingcited by 1
  5. 2025ColLab: A Collaborative Spatial Progressive Data Engine for Referring Expression Comprehension and Generationcited by 0