# PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers

arXiv: https://arxiv.org/abs/2206.02066 · HTML: https://mlsanity.com/paper/2206.02066/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 61 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1606.02147/) (2016)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)

## Built on by (most cited first, 30 of 84)

- [ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer](https://mlsanity.com/paper/2410.00086/) (2024)
- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [PEM: Prototype-based Efficient MaskFormer for Image Segmentation](https://mlsanity.com/paper/2402.19422/) (2024)
- [Frequency-Adaptive Dilated Convolution for Semantic Segmentation](https://mlsanity.com/paper/2403.05369/) (2024)
- [SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentation](https://mlsanity.com/paper/2312.17071/) (2023)
- [Frequency Dynamic Convolution for Dense Image Prediction](https://mlsanity.com/paper/2503.18783/) (2025)
- [EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality](https://mlsanity.com/paper/2411.15241/) (2024)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [The 2nd Workshop on Maritime Computer Vision (MaCVi) 2024](https://mlsanity.com/paper/2311.14762/) (2023)
- [Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator](https://mlsanity.com/paper/2403.03468/) (2024)
- [Staircase Cascaded Fusion of Lightweight Local Pattern Recognition and Long-Range Dependencies for Structural Crack Segmentation](https://mlsanity.com/paper/2408.12815/) (2024)
- [Uncertainty-Participation Context Consistency Learning for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2412.17331/) (2024)
- [Mobile-Seed: Joint Semantic Segmentation and Boundary Detection for Mobile Robots](https://mlsanity.com/paper/2311.12651/) (2023)
- [Detecting Wildfires on UAVs with Real-time Segmentation Trained by Larger Teacher Models](https://mlsanity.com/paper/2408.10843/) (2024)
- [DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentation](https://mlsanity.com/paper/2406.03702/) (2024)
- [Golden Cudgel Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/2503.03325/) (2025)
- [Improving underwater semantic segmentation with underwater image quality attention and muti-scale aggregation attention](https://mlsanity.com/paper/2503.23422/) (2025)
- [Self-training via Metric Learning for Source-Free Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/2212.04227/) (2022)
- [An ADRC-Incorporated Stochastic Gradient Descent Algorithm for Latent Factor Analysis](https://mlsanity.com/paper/2401.07012/) (2024)
- [A Deep Semantic Segmentation Network with Semantic and Contextual Refinements](https://mlsanity.com/paper/2412.08671/) (2024)
- [Multi-Scale Foreground-Background Confidence for Out-of-Distribution Segmentation](https://mlsanity.com/paper/2412.16990/) (2024)
- [Benchmarking the Spatial Robustness of DNNs via Natural and Adversarial Localized Corruptions](https://mlsanity.com/paper/2504.01632/) (2025)
- [Boosting Semantic Segmentation with Semantic Boundaries](https://mlsanity.com/paper/2304.09427/) (2023)
- [Not All Pixels Are Equal: Learning Pixel Hardness for Semantic Segmentation](https://mlsanity.com/paper/2305.08462/) (2023)
- [MarsSeg: Mars Surface Semantic Segmentation with Multi-level Extractor and Connector](https://mlsanity.com/paper/2404.04155/) (2024)
- [The revenge of BiSeNet: Efficient Multi-Task Image Segmentation](https://mlsanity.com/paper/2404.09570/) (2024)
- [Reparameterizable Dual-Resolution Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/2406.12496/) (2024)
- [Introducing VaDA: Novel Image Segmentation Model for Maritime Object Segmentation Using New Dataset](https://mlsanity.com/paper/2407.09005/) (2024)
- [LuSeg: Efficient Negative and Positive Obstacles Segmentation via Contrast-Driven Multi-Modal Feature Fusion on the Lunar](https://mlsanity.com/paper/2503.11409/) (2025)
- [MaSS13K: A Matting-level Semantic Segmentation Benchmark](https://mlsanity.com/paper/2503.18364/) (2025)

## Abstract (arXiv)

Two-branch network architecture has shown its efficiency and effectiveness in real-time semantic segmentation tasks. However, direct fusion of high-resolution details and low-frequency context has the drawback of detailed features being easily overwhelmed by surrounding contextual information. This overshoot phenomenon limits the improvement of the segmentation accuracy of existing two-branch models. In this paper, we make a connection between Convolutional Neural Networks (CNN) and Proportional-Integral-Derivative (PID) controllers and reveal that a two-branch network is equivalent to a Proportional-Integral (PI) controller, which inherently suffers from similar overshoot issues. To alleviate this problem, we propose a novel three-branch network architecture: PIDNet, which contains three branches to parse detailed, context and boundary information, respectively, and employs boundary attention to guide the fusion of detailed and context branches. Our family of PIDNets achieve the best trade-off between inference speed and accuracy and their accuracy surpasses all the existing models with similar inference speed on the Cityscapes and CamVid datasets. Specifically, PIDNet-S achieves 78.6% mIOU with inference speed of 93.2 FPS on Cityscapes and 80.1% mIOU with speed of 153.7 FPS on CamVid.