# BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation

arXiv: https://arxiv.org/abs/2004.02147 · HTML: https://mlsanity.com/paper/2004.02147/

## Key facts

- Cited by 294 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2020.

## Built on by (most cited first, 30 of 294)

- [SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation](https://mlsanity.com/paper/2209.08575/) (2022)
- [SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting](https://mlsanity.com/paper/2403.05087/) (2024)
- [UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery](https://mlsanity.com/paper/2109.08937/) (2021)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [Rethinking BiSeNet For Real-time Semantic Segmentation](https://mlsanity.com/paper/2104.13188/) (2021)
- [FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding](https://mlsanity.com/paper/2312.02214/) (2023)
- [Improving Generalization in Federated Learning by Seeking Flat Minima](https://mlsanity.com/paper/2203.11834/) (2022)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers](https://mlsanity.com/paper/2206.02066/) (2022)
- [FaPN: Feature-aligned Pyramid Network for Dense Image Prediction](https://mlsanity.com/paper/2108.07058/) (2021)
- [Stitch it in Time: GAN-Based Facial Editing of Real Videos](https://mlsanity.com/paper/2201.08361/) (2022)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [Fine-Grained Face Swapping via Regional GAN Inversion](https://mlsanity.com/paper/2211.14068/) (2022)
- [HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting](https://mlsanity.com/paper/2312.02902/) (2023)
- [ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation](https://mlsanity.com/paper/2308.00906/) (2023)
- [GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2103.04233/) (2021)
- [FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2202.13670/) (2022)
- [EdgeSAM: Prompt-In-the-Loop Distillation for SAM](https://mlsanity.com/paper/2312.06660/) (2023)
- [You Only Segment Once: Towards Real-Time Panoptic Segmentation](https://mlsanity.com/paper/2303.14651/) (2023)
- [Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encoding](https://mlsanity.com/paper/2212.02802/) (2022)
- [Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning](https://mlsanity.com/paper/2203.02573/) (2022)
- [PEM: Prototype-based Efficient MaskFormer for Image Segmentation](https://mlsanity.com/paper/2402.19422/) (2024)
- [PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation](https://mlsanity.com/paper/2101.06175/) (2021)
- [HHAvatar: Gaussian Head Avatar with Dynamic Hairs](https://mlsanity.com/paper/2312.03029/) (2023)
- [Frequency-Adaptive Dilated Convolution for Semantic Segmentation](https://mlsanity.com/paper/2403.05369/) (2024)
- [PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model](https://mlsanity.com/paper/2204.02681/) (2022)
- [RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer](https://mlsanity.com/paper/2210.07124/) (2022)
- [SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage](https://mlsanity.com/paper/2204.03648/) (2022)
- [SphereHead: Stable 3D Full-head Synthesis with Spherical Tri-plane Representation](https://mlsanity.com/paper/2404.05680/) (2024)

## Abstract (arXiv)

The low-level details and high-level semantics are both essential to the semantic segmentation task. However, to speed up the model inference, current approaches almost always sacrifice the low-level details, which leads to a considerable accuracy decrease. We propose to treat these spatial details and categorical semantics separately to achieve high accuracy and high efficiency for realtime semantic segmentation. To this end, we propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral Segmentation Network (BiSeNet V2). This architecture involves: (i) a Detail Branch, with wide channels and shallow layers to capture low-level details and generate high-resolution feature representation; (ii) a Semantic Branch, with narrow channels and deep layers to obtain high-level semantic context. The Semantic Branch is lightweight due to reducing the channel capacity and a fast-downsampling strategy. Furthermore, we design a Guided Aggregation Layer to enhance mutual connections and fuse both types of feature representation. Besides, a booster training strategy is designed to improve the segmentation performance without any extra inference cost. Extensive quantitative and qualitative evaluations demonstrate that the proposed architecture performs favourably against a few state-of-the-art real-time semantic segmentation approaches. Specifically, for a 2,048x1,024 input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce GTX 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy.