# ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

arXiv: https://arxiv.org/abs/1803.06815 · HTML: https://mlsanity.com/paper/1803.06815/

## Key facts

- Cited by 133 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2018.

## Built on by (most cited first, 30 of 133)

- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation](https://mlsanity.com/paper/2209.08575/) (2022)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Cross-Image Relational Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2204.06986/) (2022)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models](https://mlsanity.com/paper/2210.02025/) (2022)
- [FaPN: Feature-aligned Pyramid Network for Dense Image Prediction](https://mlsanity.com/paper/2108.07058/) (2021)
- [Representation Compensation Networks for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.05402/) (2022)
- [EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2106.15277/) (2021)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)
- [Efficient Medical Image Segmentation Based on Knowledge Distillation](https://mlsanity.com/paper/2108.09987/) (2021)
- [CaraNet: Context Axial Reverse Attention Network for Segmentation of Small Medical Objects](https://mlsanity.com/paper/2301.13366/) (2023)
- [EdgeSAM: Prompt-In-the-Loop Distillation for SAM](https://mlsanity.com/paper/2312.06660/) (2023)
- [You Only Segment Once: Towards Real-Time Panoptic Segmentation](https://mlsanity.com/paper/2303.14651/) (2023)
- [SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2307.12591/) (2023)
- [PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model](https://mlsanity.com/paper/2204.02681/) (2022)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation](https://mlsanity.com/paper/2211.06866/) (2022)
- [Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms](https://mlsanity.com/paper/2103.02433/) (2021)
- [Cross-Dataset Collaborative Learning for Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2103.11351/) (2021)
- [Double Similarity Distillation for Semantic Image Segmentation](https://mlsanity.com/paper/2107.08591/) (2021)
- [Multi-Exit Semantic Segmentation Networks](https://mlsanity.com/paper/2106.03527/) (2021)
- [Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Model](https://mlsanity.com/paper/2406.19369/) (2024)
- [Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales](https://mlsanity.com/paper/2106.06418/) (2021)
- [FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/2109.00699/) (2021)
- [TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation](https://mlsanity.com/paper/2202.13393/) (2022)
- [Towards Diverse Binary Segmentation via A Simple yet General Gated Network](https://mlsanity.com/paper/2303.10396/) (2023)
- [Distilling Inter-Class Distance for Semantic Segmentation](https://mlsanity.com/paper/2205.03650/) (2022)
- [Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation](https://mlsanity.com/paper/2106.15287/) (2021)
- [Graph Attention Layer Evolves Semantic Segmentation for Road Pothole Detection: A Benchmark and Algorithms](https://mlsanity.com/paper/2109.02711/) (2021)

## Abstract (arXiv)

We introduce a fast and efficient convolutional neural network, ESPNet, for semantic segmentation of high resolution images under resource constraints. ESPNet is based on a new convolutional module, efficient spatial pyramid (ESP), which is efficient in terms of computation, memory, and power. ESPNet is 22 times faster (on a standard GPU) and 180 times smaller than the state-of-the-art semantic segmentation network PSPNet, while its category-wise accuracy is only 8% less. We evaluated ESPNet on a variety of semantic segmentation datasets including Cityscapes, PASCAL VOC, and a breast biopsy whole slide image dataset. Under the same constraints on memory and computation, ESPNet outperforms all the current efficient CNN networks such as MobileNet, ShuffleNet, and ENet on both standard metrics and our newly introduced performance metrics that measure efficiency on edge devices. Our network can process high resolution images at a rate of 112 and 9 frames per second on a standard GPU and edge device, respectively.