# Semantic Object Parsing with Graph LSTM

arXiv: https://arxiv.org/abs/1603.07063 · HTML: https://mlsanity.com/paper/1603.07063/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2016.

## Built on by (most cited first, 30 of 44)

- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Concept Decomposition for Visual Exploration and Inspiration](https://mlsanity.com/paper/2305.18203/) (2023)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Part-aware Panoptic Segmentation](https://mlsanity.com/paper/2106.06351/) (2021)
- [Spatial Dual-Modality Graph Reasoning for Key Information Extraction](https://mlsanity.com/paper/2103.14470/) (2021)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [On the Road to Online Adaptation for Semantic Image Segmentation](https://mlsanity.com/paper/2203.16195/) (2022)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance](https://mlsanity.com/paper/2110.15238/) (2021)
- [Deep Learning Technique for Human Parsing: A Survey and Outlook](https://mlsanity.com/paper/2301.00394/) (2023)
- [Transformer Scale Gate for Semantic Segmentation](https://mlsanity.com/paper/2205.07056/) (2022)
- [Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles](https://mlsanity.com/paper/2103.14098/) (2021)
- [Graph Neural Networks for Cross-Camera Data Association](https://mlsanity.com/paper/2201.06311/) (2022)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Bayesian graph convolutional neural networks via tempered MCMC](https://mlsanity.com/paper/2104.08438/) (2021)
- [Estimating the State of Epidemics Spreading with Graph Neural Networks](https://mlsanity.com/paper/2105.05060/) (2021)
- [FLOAT: Factorized Learning of Object Attributes for Improved Multi-object Multi-part Scene Parsing](https://mlsanity.com/paper/2203.16168/) (2022)
- [Survey of Image Based Graph Neural Networks](https://mlsanity.com/paper/2106.06307/) (2021)
- [GNNSampler: Bridging the Gap between Sampling Algorithms of GNN and Hardware](https://mlsanity.com/paper/2108.11571/) (2021)
- [MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection](https://mlsanity.com/paper/2204.01349/) (2022)
- [Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer](https://mlsanity.com/paper/2101.10620/) (2021)
- [SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images](https://mlsanity.com/paper/2407.09686/) (2024)
- [End-to-end One-shot Human Parsing](https://mlsanity.com/paper/2105.01241/) (2021)
- [A Comprehensive Survey on Graph Summarization with Graph Neural Networks](https://mlsanity.com/paper/2302.06114/) (2023)
- [GROWL: Group Detection With Link Prediction](https://mlsanity.com/paper/2111.04397/) (2021)
- [StreamMOTP: Streaming and Unified Framework for Joint 3D Multi-Object Tracking and Trajectory Prediction](https://mlsanity.com/paper/2406.19844/) (2024)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) for Remote Land-use Change Detection](https://mlsanity.com/paper/2303.14322/) (2023)
- [Optimum Output Long Short-Term Memory Cell for High-Frequency Trading Forecasting](https://mlsanity.com/paper/2304.09840/) (2023)

## Abstract (arXiv)

By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data. Particularly, instead of evenly and fixedly dividing an image to pixels or patches in existing multi-dimensional LSTM structures (e.g., Row, Grid and Diagonal LSTMs), we take each arbitrary-shaped superpixel as a semantically consistent node, and adaptively construct an undirected graph for each image, where the spatial relations of the superpixels are naturally used as edges. Constructed on such an adaptive graph topology, the Graph LSTM is more naturally aligned with the visual patterns in the image (e.g., object boundaries or appearance similarities) and provides a more economical information propagation route. Furthermore, for each optimization step over Graph LSTM, we propose to use a confidence-driven scheme to update the hidden and memory states of nodes progressively till all nodes are updated. In addition, for each node, the forgets gates are adaptively learned to capture different degrees of semantic correlation with neighboring nodes. Comprehensive evaluations on four diverse semantic object parsing datasets well demonstrate the significant superiority of our Graph LSTM over other state-of-the-art solutions.