# Instance-Specific Feature Propagation for Referring Segmentation

arXiv: https://arxiv.org/abs/2204.12109 · HTML: https://mlsanity.com/paper/2204.12109/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Modeling Context Between Objects for Referring Expression Understanding](https://mlsanity.com/paper/1608.00525/) (2016)
- [Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2108.05565/) (2021)
- [Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2003.08813/) (2020)
- [Referring Image Segmentation via Cross-Modal Progressive Comprehension](https://mlsanity.com/paper/2010.00514/) (2020)
- [Recurrent Multimodal Interaction for Referring Image Segmentation](https://mlsanity.com/paper/1703.07939/) (2017)
- [Natural Language Object Retrieval](https://mlsanity.com/paper/1511.04164/) (2015)
- [PhraseCut: Language-based Image Segmentation in the Wild](https://mlsanity.com/paper/2008.01187/) (2020)
- [Dynamic Graph Attention for Referring Expression Comprehension](https://mlsanity.com/paper/1909.08164/) (2019)
- [Boundary-Aware Feature Propagation for Scene Segmentation](https://mlsanity.com/paper/1909.00179/) (2019)
- [Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networks](https://mlsanity.com/paper/1812.04794/) (2018)
- [Linguistic Structure Guided Context Modeling for Referring Image Segmentation](https://mlsanity.com/paper/2010.00515/) (2020)
- [Dynamic Multimodal Instance Segmentation guided by natural language queries](https://mlsanity.com/paper/1807.02257/) (2018)
- [A Joint Speaker-Listener-Reinforcer Model for Referring Expressions](https://mlsanity.com/paper/1612.09542/) (2016)
- [Semantic Correlation Promoted Shape-Variant Context for Segmentation](https://mlsanity.com/paper/1909.02651/) (2019)
- [Comprehension-guided referring expressions](https://mlsanity.com/paper/1701.03439/) (2017)
- [Referring Expression Object Segmentation with Caption-Aware Consistency](https://mlsanity.com/paper/1910.04748/) (2019)
- [Feature Boosting Network For 3D Pose Estimation](https://mlsanity.com/paper/1901.04877/) (2019)

## Built on by (most cited first, 30 of 45)

- [GRES: Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2306.00968/) (2023)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions](https://mlsanity.com/paper/2308.08544/) (2023)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [VLT: Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2210.15871/) (2022)
- [Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentation](https://mlsanity.com/paper/2404.03645/) (2024)
- [Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model](https://mlsanity.com/paper/2312.12423/) (2023)
- [Towards Robust Referring Image Segmentation](https://mlsanity.com/paper/2209.09554/) (2022)
- [GREC: Generalized Referring Expression Comprehension](https://mlsanity.com/paper/2308.16182/) (2023)
- [Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation](https://mlsanity.com/paper/2305.15302/) (2023)
- [SegPoint: Segment Any Point Cloud via Large Language Model](https://mlsanity.com/paper/2407.13761/) (2024)
- [MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentation](https://mlsanity.com/paper/2512.10945/) (2025)
- [Primitive Generation and Semantic-related Alignment for Universal Zero-Shot Segmentation](https://mlsanity.com/paper/2306.11087/) (2023)
- [RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation](https://mlsanity.com/paper/2307.00997/) (2023)
- [RefMask3D: Language-Guided Transformer for 3D Referring Segmentation](https://mlsanity.com/paper/2407.18244/) (2024)
- [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025)
- [Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentation](https://mlsanity.com/paper/2408.07539/) (2024)
- [Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyond](https://mlsanity.com/paper/2308.07539/) (2023)
- [3D-GRES: Generalized 3D Referring Expression Segmentation](https://mlsanity.com/paper/2407.20664/) (2024)
- [SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation](https://mlsanity.com/paper/2406.01451/) (2024)
- [Advancing Referring Expression Segmentation Beyond Single Image](https://mlsanity.com/paper/2305.12452/) (2023)
- [Semantic-Promoted Debiasing and Background Disambiguation for Zero-Shot Instance Segmentation](https://mlsanity.com/paper/2305.13173/) (2023)
- [DeRIS: Decoupling Perception and Cognition for Enhanced Referring Image Segmentation through Loopback Synergy](https://mlsanity.com/paper/2507.01738/) (2025)
- [Self-Regularized Prototypical Network for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2210.16829/) (2022)
- [PVUW 2024 Challenge on Complex Video Understanding: Methods and Results](https://mlsanity.com/paper/2406.17005/) (2024)
- [Hierarchical Alignment-enhanced Adaptive Grounding Network for Generalized Referring Expression Comprehension](https://mlsanity.com/paper/2501.01416/) (2025)
- [Position-Aware Contrastive Alignment for Referring Image Segmentation](https://mlsanity.com/paper/2212.13419/) (2022)
- [Bring Adaptive Binding Prototypes to Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2405.15169/) (2024)
- [LSVOS Challenge Report: Large-scale Complex and Long Video Object Segmentation](https://mlsanity.com/paper/2409.05847/) (2024)
- [GREx: Generalized Referring Expression Segmentation, Comprehension, and Generation](https://mlsanity.com/paper/2601.05244/) (2026)

## Abstract (arXiv)

Referring segmentation aims to generate a segmentation mask for the target instance indicated by a natural language expression. There are typically two kinds of existing methods: one-stage methods that directly perform segmentation on the fused vision and language features; and two-stage methods that first utilize an instance segmentation model for instance proposal and then select one of these instances via matching them with language features. In this work, we propose a novel framework that simultaneously detects the target-of-interest via feature propagation and generates a fine-grained segmentation mask. In our framework, each instance is represented by an Instance-Specific Feature (ISF), and the target-of-referring is identified by exchanging information among all ISFs using our proposed Feature Propagation Module (FPM). Our instance-aware approach learns the relationship among all objects, which helps to better locate the target-of-interest than one-stage methods. Comparing to two-stage methods, our approach collaboratively and interactively utilizes both vision and language information for synchronous identification and segmentation. In the experimental tests, our method outperforms previous state-of-the-art methods on all three RefCOCO series datasets.