# Exploring Context with Deep Structured models for Semantic Segmentation

arXiv: https://arxiv.org/abs/1603.03183 · HTML: https://mlsanity.com/paper/1603.03183/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2016.

## Built on by (most cited first, 6 of 6)

- [CTNet: Context-based Tandem Network for Semantic Segmentation](https://mlsanity.com/paper/2104.09805/) (2021)
- [Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2305.03944/) (2023)
- [Self-Adversarial Disentangling for Specific Domain Adaptation](https://mlsanity.com/paper/2108.03553/) (2021)
- [Empirical Study of Multi-Task Hourglass Model for Semantic Segmentation Task](https://mlsanity.com/paper/2105.13531/) (2021)
- [Interpretable Small Training Set Image Segmentation Network Originated from Multi-Grid Variational Model](https://mlsanity.com/paper/2306.14097/) (2023)
- [Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation](https://mlsanity.com/paper/2207.09162/) (2022)

## Abstract (arXiv)

State-of-the-art semantic image segmentation methods are mostly based on training deep convolutional neural networks (CNNs). In this work, we proffer to improve semantic segmentation with the use of contextual information. In particular, we explore `patch-patch' context and `patch-background' context in deep CNNs. We formulate deep structured models by combining CNNs and Conditional Random Fields (CRFs) for learning the patch-patch context between image regions. Specifically, we formulate CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied in order to avoid repeated expensive CRF inference during the course of back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image inputs and sliding pyramid pooling is very effective for improving performance. We perform comprehensive evaluation of the proposed method. We achieve new state-of-the-art performance on a number of challenging semantic segmentation datasets including NYUDv2, PASCAL-VOC2012, Cityscapes, PASCAL-Context, SUN-RGBD, SIFT-flow, and KITTI datasets. Particularly, we report an intersection-over-union score of 77.8 on the PASCAL-VOC2012 dataset.