# PP-HumanSeg: Connectivity-Aware Portrait Segmentation with a Large-Scale Teleconferencing Video Dataset

arXiv: https://arxiv.org/abs/2112.07146 · HTML: https://mlsanity.com/paper/2112.07146/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 27 of 27)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [Fast-SCNN: Fast Semantic Segmentation Network](https://mlsanity.com/paper/1902.04502/) (2019)
- [SegFix: Model-Agnostic Boundary Refinement for Segmentation](https://mlsanity.com/paper/2007.04269/) (2020)
- [Boundary-preserving Mask R-CNN](https://mlsanity.com/paper/2007.08921/) (2020)
- [Region Mutual Information Loss for Semantic Segmentation](https://mlsanity.com/paper/1910.12037/) (2019)
- [EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow](https://mlsanity.com/paper/2109.09406/) (2021)
- [PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation](https://mlsanity.com/paper/2101.06175/) (2021)
- [ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time](https://mlsanity.com/paper/1805.04554/) (2018)
- [Face.evoLVe: A High-Performance Face Recognition Library](https://mlsanity.com/paper/2107.08621/) (2021)
- [Guided Upsampling Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1807.07466/) (2018)
- [Personalized Image Semantic Segmentation](https://mlsanity.com/paper/2107.13978/) (2021)
- [Flow-based Video Segmentation for Human Head and Shoulders](https://mlsanity.com/paper/2104.09752/) (2021)

## Built on by (most cited first, 9 of 9)

- [PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model](https://mlsanity.com/paper/2204.02681/) (2022)
- [PP-Matting: High-Accuracy Natural Image Matting](https://mlsanity.com/paper/2204.09433/) (2022)
- [EISeg: An Efficient Interactive Segmentation Tool based on PaddlePaddle](https://mlsanity.com/paper/2210.08788/) (2022)
- [Test-time Adaptation vs. Training-time Generalization: A Case Study in Human Instance Segmentation using Keypoints Estimation](https://mlsanity.com/paper/2212.06242/) (2022)
- [RAIS: Robust and Accurate Interactive Segmentation via Continual Learning](https://mlsanity.com/paper/2210.10984/) (2022)
- [IFAST: Weakly Supervised Interpretable Face Anti-spoofing from Single-shot Binocular NIR Images](https://mlsanity.com/paper/2309.17399/) (2023)
- [MHRGait: Gait Recognition from Momentum Human Rig Pose](https://mlsanity.com/paper/2607.29083/) (2026)
- [EasyPortrait -- Face Parsing and Portrait Segmentation Dataset](https://mlsanity.com/paper/2304.13509/) (2023)
- [Towards Imbalanced Motion: Part-Decoupling Network for Video Portrait Segmentation](https://mlsanity.com/paper/2307.16565/) (2023)

## Abstract (arXiv)

As the COVID-19 pandemic rampages across the world, the demands of video conferencing surge. To this end, real-time portrait segmentation becomes a popular feature to replace backgrounds of conferencing participants. While feature-rich datasets, models and algorithms have been offered for segmentation that extract body postures from life scenes, portrait segmentation has yet not been well covered in a video conferencing context. To facilitate the progress in this field, we introduce an open-source solution named PP-HumanSeg. This work is the first to construct a large-scale video portrait dataset that contains 291 videos from 23 conference scenes with 14K fine-labeled frames and extensions to multi-camera teleconferencing. Furthermore, we propose a novel Semantic Connectivity-aware Learning (SCL) for semantic segmentation, which introduces a semantic connectivity-aware loss to improve the quality of segmentation results from the perspective of connectivity. And we propose an ultra-lightweight model with SCL for practical portrait segmentation, which achieves the best trade-off between IoU and the speed of inference. Extensive evaluations on our dataset demonstrate the superiority of SCL and our model. The source code is available at https://github.com/PaddlePaddle/PaddleSeg.