# Dynamic-structured Semantic Propagation Network

arXiv: https://arxiv.org/abs/1803.06067 · HTML: https://mlsanity.com/paper/1803.06067/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2018.

## Built on by (most cited first, 30 of 38)

- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Hyperbolic Image Segmentation](https://mlsanity.com/paper/2203.05898/) (2022)
- [Deep Hierarchical Semantic Segmentation](https://mlsanity.com/paper/2203.14335/) (2022)
- [CTNet: Context-based Tandem Network for Semantic Segmentation](https://mlsanity.com/paper/2104.09805/) (2021)
- [Polarized Self-Attention: Towards High-quality Pixel-wise Regression](https://mlsanity.com/paper/2107.00782/) (2021)
- [INTERN: A New Learning Paradigm Towards General Vision](https://mlsanity.com/paper/2111.08687/) (2021)
- [LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning](https://mlsanity.com/paper/2309.13556/) (2023)
- [Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation](https://mlsanity.com/paper/2102.10935/) (2021)
- [Channelized Axial Attention for Semantic Segmentation -- Considering Channel Relation within Spatial Attention for Semantic Segmentation](https://mlsanity.com/paper/2101.07434/) (2021)
- [A Unified Efficient Pyramid Transformer for Semantic Segmentation](https://mlsanity.com/paper/2107.14209/) (2021)
- [Multi-domain semantic segmentation with overlapping labels](https://mlsanity.com/paper/2108.11224/) (2021)
- [Learning Semantic Segmentation from Multiple Datasets with Label Shifts](https://mlsanity.com/paper/2202.14030/) (2022)
- [Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity](https://mlsanity.com/paper/2303.05689/) (2023)
- [Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention](https://mlsanity.com/paper/2304.03282/) (2023)
- [Flattening the Parent Bias: Hierarchical Semantic Segmentation in the Poincar'e Ball](https://mlsanity.com/paper/2404.03778/) (2024)
- [Global Aggregation then Local Distribution for Scene Parsing](https://mlsanity.com/paper/2107.13154/) (2021)
- [COLA: COarse-LAbel multi-source LiDAR semantic segmentation for autonomous driving](https://mlsanity.com/paper/2311.03017/) (2023)
- [ORDNet: Capturing Omni-Range Dependencies for Scene Parsing](https://mlsanity.com/paper/2101.03929/) (2021)
- [Multi-dataset Pretraining: A Unified Model for Semantic Segmentation](https://mlsanity.com/paper/2106.04121/) (2021)
- [Automatic universal taxonomies for multi-domain semantic segmentation](https://mlsanity.com/paper/2207.08445/) (2022)
- [Confident AI](https://mlsanity.com/paper/2202.05957/) (2022)
- [Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer](https://mlsanity.com/paper/2101.10620/) (2021)
- [Weakly supervised training of universal visual concepts for multi-domain semantic segmentation](https://mlsanity.com/paper/2212.10340/) (2022)
- [COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets](https://mlsanity.com/paper/2202.06884/) (2022)
- [U-HRNet: Delving into Improving Semantic Representation of High Resolution Network for Dense Prediction](https://mlsanity.com/paper/2210.07140/) (2022)
- [Training Semantic Segmentation on Heterogeneous Datasets](https://mlsanity.com/paper/2301.07634/) (2023)
- [Traffic Scene Parsing through the TSP6K Dataset](https://mlsanity.com/paper/2303.02835/) (2023)
- [Hierarchical Pyramid Representations for Semantic Segmentation](https://mlsanity.com/paper/2104.01792/) (2021)
- [DCANet: Dense Context-Aware Network for Semantic Segmentation](https://mlsanity.com/paper/2104.02533/) (2021)
- [Boundary-aware Graph Reasoning for Semantic Segmentation](https://mlsanity.com/paper/2108.03791/) (2021)

## Abstract (arXiv)

Semantic concept hierarchy is still under-explored for semantic segmentation due to the inefficiency and complicated optimization of incorporating structural inference into dense prediction. This lack of modeling semantic correlations also makes prior works must tune highly-specified models for each task due to the label discrepancy across datasets. It severely limits the generalization capability of segmentation models for open set concept vocabulary and annotation utilization. In this paper, we propose a Dynamic-Structured Semantic Propagation Network (DSSPN) that builds a semantic neuron graph by explicitly incorporating the semantic concept hierarchy into network construction. Each neuron represents the instantiated module for recognizing a specific type of entity such as a super-class (e.g. food) or a specific concept (e.g. pizza). During training, DSSPN performs the dynamic-structured neuron computation graph by only activating a sub-graph of neurons for each image in a principled way. A dense semantic-enhanced neural block is proposed to propagate the learned knowledge of all ancestor neurons into each fine-grained child neuron for feature evolving. Another merit of such semantic explainable structure is the ability of learning a unified model concurrently on diverse datasets by selectively activating different neuron sub-graphs for each annotation at each step. Extensive experiments on four public semantic segmentation datasets (i.e. ADE20K, COCO-Stuff, Cityscape and Mapillary) demonstrate the superiority of our DSSPN over state-of-the-art segmentation models. Moreoever, we demonstrate a universal segmentation model that is jointly trained on diverse datasets can surpass the performance of the common fine-tuning scheme for exploiting multiple domain knowledge.