mlsanityData to 7 Oct 2026

Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentation

arXiv 2105.01839 ↗First posted 5 May 2021cs.CVcited by 111

Not on the map. Why?

Abstract (arXiv)

Recently, referring image segmentation has aroused widespread interest. Previous methods perform the multi-modal fusion between language and vision at the decoding side of the network. And, linguistic feature interacts with visual feature of each scale separately, which ignores the continuous guidance of language to multi-scale visual features. In this work, we propose an encoder fusion network (EFN), which transforms the visual encoder into a multi-modal feature learning network, and uses language to refine the multi-modal features progressively. Moreover, a co-attention mechanism is embedded in the EFN to realize the parallel update of multi-modal features, which can promote the consistent of the cross-modal information representation in the semantic space. Finally, we propose a boundary enhancement module (BEM) to make the network pay more attention to the fine structure. The experiment results on four benchmark datasets demonstrate that the proposed approach achieves the state-of-the-art performance under different evaluation metrics without any post-processing.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

32 of 45 references.

References · 32

  1. 2017Mask R-CNNcited by 7,259
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2016Wide Residual Networkscited by 3,057
  6. 2016Pyramid Scene Parsing Networkcited by 2,472
  7. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  8. 2015Spatial Transformer Networkscited by 1,423
  9. 2016Modeling Context in Referring Expressionscited by 1,054
  10. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  11. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  12. 2018Path Aggregation Network for Instance Segmentationcited by 796
20 more
  1. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  2. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  3. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  4. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  5. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  6. 2016Segmentation from Natural Language Expressionscited by 257
  7. 2019Asymmetric Non-local Neural Networks for Semantic Segmentationcited by 196
  8. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 194
  9. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  10. 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
  11. 2015Natural Language Object Retrievalcited by 124
  12. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 120
  13. 2019A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehensioncited by 111
  14. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  15. 2018Dynamic Multimodal Instance Segmentation guided by natural language queriescited by 83
  16. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  17. 2016A Joint Speaker-Listener-Reinforcer Model for Referring Expressionscited by 74
  18. 2020Boundary-preserving Mask R-CNNcited by 74
  19. 2017Comprehension-guided referring expressionscited by 52
  20. 2019Referring Expression Object Segmentation with Caption-Aware Consistencycited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 12; 2023: 49; 2024: 25; 2025: 14; 2026: 7.

111 papers.

Sort:

By area:

  1. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  2. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  3. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  4. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  5. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  6. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  7. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 153
  8. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  9. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  10. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 126
  11. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  12. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
38 more
  1. 2022Changer: Feature Interaction is What You Need for Change Detectioncited by 78
  2. 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
  3. 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
  4. 2023Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentationcited by 61
  5. 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
  6. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  7. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  8. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  9. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 53
  10. 2022Towards Robust Referring Image Segmentationcited by 49
  11. 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
  12. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  13. 2023Mask Grounding for Referring Image Segmentationcited by 40
  14. 2024ReMamber: Referring Image Segmentation with Mamba Twistercited by 38
  15. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  16. 2023Language Adaptive Weight Generation for Multi-task Visual Groundingcited by 35
  17. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  18. 2023Universal Segmentation at Arbitrary Granularity with Language Instructioncited by 29
  19. 2022CoupAlign: Coupling Word-Pixel with Sentence-Mask Alignments for Referring Image Segmentationcited by 28
  20. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  21. 2023Text Promptable Surgical Instrument Segmentation with Vision-Language Modelscited by 27
  22. 2023Temporal Collection and Distribution for Referring Video Object Segmentationcited by 27
  23. 2023Shatter and Gather: Learning Referring Image Segmentation with Text Supervisioncited by 25
  24. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 24
  25. 2023Context Disentangling and Prototype Inheriting for Robust Visual Groundingcited by 23
  26. 2024Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentationcited by 23
  27. 2023DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformerscited by 22
  28. 2023Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Cluttercited by 22
  29. 2023UniRef++: Segment Every Reference Object in Spatial and Temporal Spacescited by 22
  30. 2025Pixel-SAIL: Single Transformer For Pixel-Grounded Understandingcited by 22
  31. 2025Multimodal Referring Segmentation: A Surveycited by 22
  32. 2023Unveiling Parts Beyond Objects:Towards Finer-Granularity Referring Expression Segmentationcited by 21
  33. 2024An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Groundingcited by 21
  34. 2024Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentationcited by 21
  35. 2023Extending CLIP's Image-Text Alignment to Referring Image Segmentationcited by 19
  36. 2023Meta Compositional Referring Expression Segmentationcited by 18
  37. 2021Towards Language-guided Visual Recognition via Dynamic Convolutionscited by 17
  38. 2023Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentationcited by 16

50 most cited of 111 All 111 (JSON)