# Semantic Correlation Promoted Shape-Variant Context for Segmentation

arXiv: https://arxiv.org/abs/1909.02651 · HTML: https://mlsanity.com/paper/1909.02651/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2019.

## Built on by (most cited first, 30 of 61)

- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2108.05565/) (2021)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [VLT: Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2210.15871/) (2022)
- [InverseForm: A Loss Function for Structured Boundary-Aware Segmentation](https://mlsanity.com/paper/2104.02745/) (2021)
- [SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation](https://mlsanity.com/paper/2305.17011/) (2023)
- [Learning Statistical Texture for Semantic Segmentation](https://mlsanity.com/paper/2103.04133/) (2021)
- [Mining Contextual Information Beyond Image for Semantic Segmentation](https://mlsanity.com/paper/2108.11819/) (2021)
- [Instance-Specific Feature Propagation for Referring Segmentation](https://mlsanity.com/paper/2204.12109/) (2022)
- [CTNet: Context-based Tandem Network for Semantic Segmentation](https://mlsanity.com/paper/2104.09805/) (2021)
- [ISNet: Integrate Image-Level and Semantic-Level Context for Semantic Segmentation](https://mlsanity.com/paper/2108.12382/) (2021)
- [Polarized Self-Attention: Towards High-quality Pixel-wise Regression](https://mlsanity.com/paper/2107.00782/) (2021)
- [Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation](https://mlsanity.com/paper/2305.15302/) (2023)
- [StructToken : Rethinking Semantic Segmentation with Structural Prior](https://mlsanity.com/paper/2203.12612/) (2022)
- [Primitive Generation and Semantic-related Alignment for Universal Zero-Shot Segmentation](https://mlsanity.com/paper/2306.11087/) (2023)
- [Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Images](https://mlsanity.com/paper/2106.15754/) (2021)
- [Prototype Adaption and Projection for Few- and Zero-shot 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2305.14335/) (2023)
- [Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation](https://mlsanity.com/paper/2207.10436/) (2022)
- [Learning Local and Global Temporal Contexts for Video Semantic Segmentation](https://mlsanity.com/paper/2204.03330/) (2022)
- [Prompt-Matched Semantic Segmentation](https://mlsanity.com/paper/2208.10159/) (2022)
- [Double Similarity Distillation for Semantic Image Segmentation](https://mlsanity.com/paper/2107.08591/) (2021)
- [Channelized Axial Attention for Semantic Segmentation -- Considering Channel Relation within Spatial Attention for Semantic Segmentation](https://mlsanity.com/paper/2101.07434/) (2021)
- [CAR: Class-aware Regularizations for Semantic Segmentation](https://mlsanity.com/paper/2203.07160/) (2022)
- [Towards Enhancing Fine-grained Details for Image Matting](https://mlsanity.com/paper/2101.09095/) (2021)
- [Adversarial Shape Learning for Building Extraction in VHR Remote Sensing Images](https://mlsanity.com/paper/2102.11262/) (2021)
- [Knowledge-aware Deep Framework for Collaborative Skin Lesion Segmentation and Melanoma Recognition](https://mlsanity.com/paper/2106.03455/) (2021)
- [M2IOSR: Maximal Mutual Information Open Set Recognition](https://mlsanity.com/paper/2108.02373/) (2021)

## Abstract (arXiv)

Context is essential for semantic segmentation. Due to the diverse shapes of objects and their complex layout in various scene images, the spatial scales and shapes of contexts for different objects have very large variation. It is thus ineffective or inefficient to aggregate various context information from a predefined fixed region. In this work, we propose to generate a scale- and shape-variant semantic mask for each pixel to confine its contextual region. To this end, we first propose a novel paired convolution to infer the semantic correlation of the pair and based on that to generate a shape mask. Using the inferred spatial scope of the contextual region, we propose a shape-variant convolution, of which the receptive field is controlled by the shape mask that varies with the appearance of input. In this way, the proposed network aggregates the context information of a pixel from its semantic-correlated region instead of a predefined fixed region. Furthermore, this work also proposes a labeling denoising model to reduce wrong predictions caused by the noisy low-level features. Without bells and whistles, the proposed segmentation network achieves new state-of-the-arts consistently on the six public segmentation datasets.