# Bilateral Network with Residual U-blocks and Dual-Guided Attention for Real-time Semantic Segmentation

arXiv: https://arxiv.org/abs/2310.20305 · HTML: https://mlsanity.com/paper/2310.20305/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 25 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation](https://mlsanity.com/paper/1803.06815/) (2018)
- [Fast-SCNN: Fast Semantic Segmentation Network](https://mlsanity.com/paper/1902.04502/) (2019)
- [Rethinking BiSeNet For Real-time Semantic Segmentation](https://mlsanity.com/paper/2104.13188/) (2021)
- [Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks](https://mlsanity.com/paper/2105.02358/) (2021)
- [DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1904.02216/) (2019)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model](https://mlsanity.com/paper/2204.02681/) (2022)
- [RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer](https://mlsanity.com/paper/2210.07124/) (2022)
- [Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1809.06323/) (2018)
- [S2-FPN: Scale-ware Strip Attention Guided Feature Pyramid Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/2206.07298/) (2022)

## Built on by (most cited first, 1 of 1)

- [Cross-Layer Attentive Feature Upsampling for Low-latency Semantic Segmentation](https://mlsanity.com/paper/2601.01167/) (2026)

## Abstract (arXiv)

When some application scenarios need to use semantic segmentation technology, like automatic driving, the primary concern comes to real-time performance rather than extremely high segmentation accuracy. To achieve a good trade-off between speed and accuracy, two-branch architecture has been proposed in recent years. It treats spatial information and semantics information separately which allows the model to be composed of two networks both not heavy. However, the process of fusing features with two different scales becomes a performance bottleneck for many nowaday two-branch models. In this research, we design a new fusion mechanism for two-branch architecture which is guided by attention computation. To be precise, we use the Dual-Guided Attention (DGA) module we proposed to replace some multi-scale transformations with the calculation of attention which means we only use several attention layers of near linear complexity to achieve performance comparable to frequently-used multi-layer fusion. To ensure that our module can be effective, we use Residual U-blocks (RSU) to build one of the two branches in our networks which aims to obtain better multi-scale features. Extensive experiments on Cityscapes and CamVid dataset show the effectiveness of our method.