# EfficientFCN: Holistically-guided Decoding for Semantic Segmentation

arXiv: https://arxiv.org/abs/2008.10487 · HTML: https://mlsanity.com/paper/2008.10487/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2020.

## Built on by (most cited first, 15 of 15)

- [SegViT: Semantic Segmentation with Plain Vision Transformers](https://mlsanity.com/paper/2210.05844/) (2022)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks](https://mlsanity.com/paper/2303.15710/) (2023)
- [RGB-T Semantic Segmentation with Location, Activation, and Sharpening](https://mlsanity.com/paper/2210.14530/) (2022)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [SegViTv2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers](https://mlsanity.com/paper/2306.06289/) (2023)
- [Fully Transformer Networks for Semantic Image Segmentation](https://mlsanity.com/paper/2106.04108/) (2021)
- [Learning Local and Global Temporal Contexts for Video Semantic Segmentation](https://mlsanity.com/paper/2204.03330/) (2022)
- [Self-Regulation for Semantic Segmentation](https://mlsanity.com/paper/2108.09702/) (2021)
- [A Unified Efficient Pyramid Transformer for Semantic Segmentation](https://mlsanity.com/paper/2107.14209/) (2021)
- [MMSFormer: Multimodal Transformer for Material and Semantic Segmentation](https://mlsanity.com/paper/2309.04001/) (2023)
- [Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/2107.14428/) (2021)
- [Pyramid Fusion Transformer for Semantic Segmentation](https://mlsanity.com/paper/2201.04019/) (2022)
- [Feature Selective Transformer for Semantic Image Segmentation](https://mlsanity.com/paper/2203.14124/) (2022)
- [Feature Reuse and Fusion for Real-time Semantic segmentation](https://mlsanity.com/paper/2105.12964/) (2021)

## Abstract (arXiv)

Both performance and efficiency are important to semantic segmentation. State-of-the-art semantic segmentation algorithms are mostly based on dilated Fully Convolutional Networks (dilatedFCN), which adopt dilated convolutions in the backbone networks to extract high-resolution feature maps for achieving high-performance segmentation performance. However, due to many convolution operations are conducted on the high-resolution feature maps, such dilatedFCN-based methods result in large computational complexity and memory consumption. To balance the performance and efficiency, there also exist encoder-decoder structures that gradually recover the spatial information by combining multi-level feature maps from the encoder. However, the performances of existing encoder-decoder methods are far from comparable with the dilatedFCN-based methods. In this paper, we propose the EfficientFCN, whose backbone is a common ImageNet pre-trained network without any dilated convolution. A holistically-guided decoder is introduced to obtain the high-resolution semantic-rich feature maps via the multi-scale features from the encoder. The decoding task is converted to novel codebook generation and codeword assembly task, which takes advantages of the high-level and low-level features from the encoder. Such a framework achieves comparable or even better performance than state-of-the-art methods with only 1/3 of the computational cost. Extensive experiments on PASCAL Context, PASCAL VOC, ADE20K validate the effectiveness of the proposed EfficientFCN.