# Semantic Diffusion Network for Semantic Segmentation

arXiv: https://arxiv.org/abs/2302.02057 · HTML: https://mlsanity.com/paper/2302.02057/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 75 references that are in the map.
- First posted on arXiv 4 Feb 2023.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [Conditional Random Fields as Recurrent Neural Networks](https://mlsanity.com/paper/1502.03240/) (2015)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)

## Built on by (most cited first, 30 of 30)

- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation](https://mlsanity.com/paper/2309.14303/) (2023)
- [Improving 2D Feature Representations by 3D-Aware Fine-Tuning](https://mlsanity.com/paper/2407.20229/) (2024)
- [FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators](https://mlsanity.com/paper/2310.03420/) (2023)
- [Unified Language-driven Zero-shot Domain Adaptation](https://mlsanity.com/paper/2404.07155/) (2024)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [Scaling Properties of Diffusion Models for Perceptual Tasks](https://mlsanity.com/paper/2411.08034/) (2024)
- [Learnable Differencing Center for Nighttime Depth Perception](https://mlsanity.com/paper/2306.14538/) (2023)
- [Formulating Discrete Probability Flow Through Optimal Transport](https://mlsanity.com/paper/2311.03886/) (2023)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [Hyperbolic Diffusion Recommender Model](https://mlsanity.com/paper/2504.01541/) (2025)
- [DiffUCD:Unsupervised Hyperspectral Image Change Detection with Semantic Correlation Diffusion Model](https://mlsanity.com/paper/2305.12410/) (2023)
- [A Refreshed Similarity-based Upsampler for Direct High-Ratio Feature Upsampling](https://mlsanity.com/paper/2407.02283/) (2024)
- [Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views](https://mlsanity.com/paper/2509.01250/) (2025)
- [RARE: Refine Any Registration of Pairwise Point Clouds via Zero-Shot Learning](https://mlsanity.com/paper/2507.19950/) (2025)
- [Equipping Vision Foundation Model with Mixture of Experts for Out-of-Distribution Detection](https://mlsanity.com/paper/2510.10584/) (2025)
- [Boosting Semantic Segmentation with Semantic Boundaries](https://mlsanity.com/paper/2304.09427/) (2023)
- [ContourDiff: Unpaired Medical Image Translation with Structural Consistency](https://mlsanity.com/paper/2403.10786/) (2024)
- [From Local Binary Patterns to Pixel Difference Networks for Efficient Visual Representation Learning](https://mlsanity.com/paper/2303.08414/) (2023)
- [Adversarially Domain-adaptive Latent Diffusion for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2412.16859/) (2024)
- [PnPNet: Pull-and-Push Networks for Volumetric Segmentation with Boundary Confusion](https://mlsanity.com/paper/2312.08323/) (2023)
- [Unleashing Diffusion and State Space Models for Medical Image Segmentation](https://mlsanity.com/paper/2506.12747/) (2025)
- [Deeply Dual Supervised learning for melanoma recognition](https://mlsanity.com/paper/2508.01994/) (2025)
- [Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsampling](https://mlsanity.com/paper/2508.06805/) (2025)
- [DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentation](https://mlsanity.com/paper/2508.06816/) (2025)
- [VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagation](https://mlsanity.com/paper/2508.06819/) (2025)
- [A Multimodal Cross-View Model for Predicting Postoperative Neck Pain in Cervical Spondylosis Patients](https://mlsanity.com/paper/2509.02256/) (2025)
- [Generative Point Tracking with Flow Matching](https://mlsanity.com/paper/2510.20951/) (2025)
- [Modular Diffusion Models for Structured Visual Recognition](https://mlsanity.com/paper/2606.22702/) (2026)
- [Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2608.22679/) (2026)

## Abstract (arXiv)

Precise and accurate predictions over boundary areas are essential for semantic segmentation. However, the commonly-used convolutional operators tend to smooth and blur local detail cues, making it difficult for deep models to generate accurate boundary predictions. In this paper, we introduce an operator-level approach to enhance semantic boundary awareness, so as to improve the prediction of the deep semantic segmentation model. Specifically, we first formulate the boundary feature enhancement as an anisotropic diffusion process. We then propose a novel learnable approach called semantic diffusion network (SDN) to approximate the diffusion process, which contains a parameterized semantic difference convolution operator followed by a feature fusion module. Our SDN aims to construct a differentiable mapping from the original feature to the inter-class boundary-enhanced feature. The proposed SDN is an efficient and flexible module that can be easily plugged into existing encoder-decoder segmentation models. Extensive experiments show that our approach can achieve consistent improvements over several typical and state-of-the-art segmentation baseline models on challenging public benchmarks. The code will be released soon.