# Joint Semantic Segmentation and Boundary Detection using Iterative Pyramid Contexts

arXiv: https://arxiv.org/abs/2004.07684 · HTML: https://mlsanity.com/paper/2004.07684/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2020.

## Built on by (most cited first, 30 of 51)

- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation](https://mlsanity.com/paper/2209.08575/) (2022)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Uncertainty-aware Joint Salient Object and Camouflaged Object Detection](https://mlsanity.com/paper/2104.02628/) (2021)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Active Boundary Loss for Semantic Segmentation](https://mlsanity.com/paper/2102.02696/) (2021)
- [RINDNet: Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth](https://mlsanity.com/paper/2108.00616/) (2021)
- [Semantic Diffusion Network for Semantic Segmentation](https://mlsanity.com/paper/2302.02057/) (2023)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Learning Local and Global Temporal Contexts for Video Semantic Segmentation](https://mlsanity.com/paper/2204.03330/) (2022)
- [AerialFormer: Multi-resolution Transformer for Aerial Image Segmentation](https://mlsanity.com/paper/2306.06842/) (2023)
- [Semi-supervised Multi-task Learning for Semantics and Depth](https://mlsanity.com/paper/2110.07197/) (2021)
- [DuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation](https://mlsanity.com/paper/2212.11677/) (2022)
- [AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation](https://mlsanity.com/paper/2409.20398/) (2024)
- [Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation](https://mlsanity.com/paper/2102.10935/) (2021)
- [FAKD: Feature Augmented Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2208.14143/) (2022)
- [Learning to Visually Localize Sound Sources from Mixtures without Prior Source Knowledge](https://mlsanity.com/paper/2403.17420/) (2024)
- [Low-Resolution Self-Attention for Semantic Segmentation](https://mlsanity.com/paper/2310.05026/) (2023)
- [Vanishing-Point-Guided Video Semantic Segmentation of Driving Scenes](https://mlsanity.com/paper/2401.15261/) (2024)
- [Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation](https://mlsanity.com/paper/2203.15259/) (2022)
- [FsaNet: Frequency Self-attention for Semantic Segmentation](https://mlsanity.com/paper/2211.15595/) (2022)
- [Self-supervised Hypergraphs for Learning Multiple World Interpretations](https://mlsanity.com/paper/2308.07615/) (2023)
- [E-CRF: Embedded Conditional Random Field for Boundary-caused Class Weights Confusion in Semantic Segmentation](https://mlsanity.com/paper/2112.07106/) (2021)
- [SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning](https://mlsanity.com/paper/2111.12358/) (2021)
- [Towards Complex Backgrounds: A Unified Difference-Aware Decoder for Binary Segmentation](https://mlsanity.com/paper/2210.15156/) (2022)
- [Multi-Task Hypergraphs for Semi-supervised Learning using Earth Observations](https://mlsanity.com/paper/2308.11021/) (2023)
- [Boundary Guided Context Aggregation for Semantic Segmentation](https://mlsanity.com/paper/2110.14587/) (2021)
- [Mobile-Seed: Joint Semantic Segmentation and Boundary Detection for Mobile Robots](https://mlsanity.com/paper/2311.12651/) (2023)
- [PNT-Edge: Towards Robust Edge Detection with Noisy Labels by Learning Pixel-level Noise Transitions](https://mlsanity.com/paper/2307.14070/) (2023)
- [Understanding Dark Scenes by Contrasting Multi-Modal Observations](https://mlsanity.com/paper/2308.12320/) (2023)

## Abstract (arXiv)

In this paper, we present a joint multi-task learning framework for semantic segmentation and boundary detection. The critical component in the framework is the iterative pyramid context module (PCM), which couples two tasks and stores the shared latent semantics to interact between the two tasks. For semantic boundary detection, we propose the novel spatial gradient fusion to suppress nonsemantic edges. As semantic boundary detection is the dual task of semantic segmentation, we introduce a loss function with boundary consistency constraint to improve the boundary pixel accuracy for semantic segmentation. Our extensive experiments demonstrate superior performance over state-of-the-art works, not only in semantic segmentation but also in semantic boundary detection. In particular, a mean IoU score of 81:8% on Cityscapes test set is achieved without using coarse data or any external data for semantic segmentation. For semantic boundary detection, we improve over previous state-of-the-art works by 9.9% in terms of AP and 6:8% in terms of MF(ODS).