mlsanityData to 14 Oct 2026

One for All: One-stage Referring Expression Comprehension with Dynamic Reasoning

arXiv 2208.00361 ↗First posted 31 Jul 2022cs.CVcited by 3

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Referring Expression Comprehension (REC) is one of the most important tasks in visual reasoning that requires a model to detect the target object referred by a natural language expression. Among the proposed pipelines, the one-stage Referring Expression Comprehension (OSREC) has become the dominant trend since it merges the region proposal and selection stages. Many state-of-the-art OSREC models adopt a multi-hop reasoning strategy because a sequence of objects is frequently mentioned in a single expression which needs multi-hop reasoning to analyze the semantic relation. However, one unsolved issue of these models is that the number of reasoning steps needs to be pre-defined and fixed before inference, ignoring the varying complexity of expressions. In this paper, we propose a Dynamic Multi-step Reasoning Network, which allows the reasoning steps to be dynamically adjusted based on the reasoning state and expression complexity. Specifically, we adopt a Transformer module to memorize & process the reasoning state and a Reinforcement Learning strategy to dynamically infer the reasoning steps. The work achieves the state-of-the-art performance or significant improvements on several REC datasets, ranging from RefCOCO (+, g) with short expressions, to Ref-Reasoning, a dataset with long and complex compositional expressions.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 57 references that are in the map. First posted on arXiv 31 Jul 2022.

Builds on

48 of 57 references.

References · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2022A ConvNet for the 2020scited by 3,579
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  8. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,630
  9. 2016Modeling Context in Referring Expressionscited by 1,063
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  12. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
36 more
  1. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  2. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  3. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  4. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 251
  5. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  6. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  7. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  8. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 140
  9. 2015Learning Deep Structure-Preserving Image-Text Embeddingscited by 139
  10. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 138
  11. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  12. 2017Learning Two-Branch Neural Networks for Image-Text Matching Taskscited by 119
  13. 2019A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehensioncited by 111
  14. 2019Dynamic Graph Attention for Referring Expression Comprehensioncited by 111
  15. 2019Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQAcited by 107
  16. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 105
  17. 2018Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networkscited by 102
  18. 2022Inception Transformercited by 99
  19. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  20. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
  21. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 83
  22. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  23. 2016A Joint Speaker-Listener-Reinforcer Model for Referring Expressionscited by 74
  24. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 71
  25. 2019Read, Watch, and Move: Reinforcement Learning for Temporally Grounding Natural Language Descriptions in Videoscited by 60
  26. 2020Graph-Structured Referring Expression Reasoning in The Wildcited by 58
  27. 2018Real-Time Referring Expression Comprehension by Single-Stage Grounding Networkcited by 56
  28. 2019Language-Conditioned Graph Networks for Relational Reasoningcited by 55
  29. 2017VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentationcited by 50
  30. 2017Conditional Image-Text Embedding Networkscited by 50
  31. 2021VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputscited by 42
  32. 2019Reasoning Visual Dialogs with Structural and Partial Observationscited by 25
  33. 2018Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captionscited by 22
  34. 2020Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCapscited by 18
  35. 2019Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressionscited by 12
  36. 2021Proposal-free One-stage Referring Expression via Grid-Word Cross-Attentioncited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2.

3 papers.

  1. 2023Transformers in Reinforcement Learning: A Surveycited by 12
  2. 2024M2IST: Multi-Modal Interactive Side-Tuning for Efficient Referring Expression Comprehensioncited by 11
  3. 2024DARA: Domain- and Relation-aware Adapters Make Parameter-efficient Tuning for Visual Groundingcited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2025RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension
  2. 2026Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Tasks
  3. 2025KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge
  4. 2026REC-RL: Referring expression counting via Gaussian and range-based reward optimization
  5. 2026VIRO: Robust and Efficient Neuro-Symbolic Reasoning with Verification for Referring Expression Comprehension
  6. 2025New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration
  7. 2024FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension
  8. 2024Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Models