mlsanityData to 14 Oct 2026

Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation

arXiv 2003.08813 ↗First posted 19 Mar 2020cs.CVcited by 204

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Referring expression comprehension (REC) and segmentation (RES) are two highly-related tasks, which both aim at identifying the referent according to a natural language expression. In this paper, we propose a novel Multi-task Collaborative Network (MCN) to achieve a joint learning of REC and RES for the first time. In MCN, RES can help REC to achieve better language-vision alignment, while REC can help RES to better locate the referent. In addition, we address a key challenge in this multi-task setup, i.e., the prediction conflict, with two innovative designs namely, Consistency Energy Maximization (CEM) and Adaptive Soft Non-Located Suppression (ASNLS). Specifically, CEM enables REC and RES to focus on similar visual regions by maximizing the consistency energy between two tasks. ASNLS supresses the response of unrelated regions in RES based on the prediction of REC. To validate our model, we conduct extensive experiments on three benchmark datasets of REC and RES, i.e., RefCOCO, RefCOCO+ and RefCOCOg. The experimental results report the significant performance gains of MCN over all existing methods, i.e., up to +7.13% for REC and +11.50% for RES over SOTA, which well confirm the validity of our model for joint REC and RES learning.

Key facts. Cited by 204 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 18; 2022: 28; 2023: 66; 2024: 48; 2025: 33; 2026: 11.

204 papers.

Sort:

By area:

  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  2. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  3. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  4. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  5. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  6. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  7. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  8. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  9. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
  10. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  11. 2022Language as Queries for Referring Video Object Segmentationcited by 182
  12. 2023Towards Open Vocabulary Learning: A Surveycited by 166
38 more
  1. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
  2. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 148
  3. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  4. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 139
  5. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  6. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 127
  7. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  8. 2021Uncertainty-aware Joint Salient Object and Camouflaged Object Detectioncited by 114
  9. 2023LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Modelscited by 113
  10. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 111
  11. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  12. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  13. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
  14. 2023Referring Multi-Object Trackingcited by 93
  15. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  16. 2023NExT-Chat: An LMM for Chat, Detection and Segmentationcited by 80
  17. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  18. 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
  19. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  20. 2023PerceptionGPT: Effectively Fusing Visual Perception into LLMcited by 59
  21. 2023SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentationcited by 57
  22. 2024LaSagnA: Language-based Segmentation Assistant for Complex Queriescited by 55
  23. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  24. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  25. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Modelcited by 53
  26. 2024Toward Visual Grounding: A Surveycited by 52
  27. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  28. 2022Towards Robust Referring Video Object Segmentation with Cyclic Relational Consensuscited by 50
  29. 2022Towards Robust Referring Image Segmentationcited by 49
  30. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  31. 2022Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Groundingcited by 42
  32. 2023GREC: Generalized Referring Expression Comprehensioncited by 42
  33. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  34. 2023See, Say, and Segment: Teaching LMMs to Overcome False Premisescited by 41
  35. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  36. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  37. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  38. 2024SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusioncited by 36

50 most cited of 204 All 204 (JSON)