# Towards Imbalanced Motion: Part-Decoupling Network for Video Portrait Segmentation

arXiv: https://arxiv.org/abs/2307.16565 · HTML: https://mlsanity.com/paper/2307.16565/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 67 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](https://mlsanity.com/paper/2003.06473/) (2020)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [SCOPS: Self-Supervised Co-Part Segmentation](https://mlsanity.com/paper/1905.01298/) (2019)
- [RVOS: End-to-End Recurrent Network for Video Object Segmentation](https://mlsanity.com/paper/1903.05612/) (2019)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Interpretable and Accurate Fine-grained Recognition via Region Grouping](https://mlsanity.com/paper/2005.10411/) (2020)
- [Background Suppression Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1911.09963/) (2019)
- [Pyramid Grafting Network for One-Stage High Resolution Saliency Detection](https://mlsanity.com/paper/2204.05041/) (2022)
- [Motion-Attentive Transition for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2003.04253/) (2020)
- [Anchor Diffusion for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/1910.10895/) (2019)
- [Learning to Segment Moving Objects](https://mlsanity.com/paper/1712.01127/) (2017)
- [Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2207.08485/) (2022)
- [F2Net: Learning to Focus on the Foreground for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2012.02534/) (2020)
- [Unsupervised Part Segmentation through Disentangling Appearance and Shape](https://mlsanity.com/paper/2105.12405/) (2021)
- [Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easier](https://mlsanity.com/paper/2112.12402/) (2021)
- [Densely Nested Top-Down Flows for Salient Object Detection](https://mlsanity.com/paper/2102.09133/) (2021)
- [EpO-Net: Exploiting Geometric Constraints on Dense Trajectories for Motion Saliency](https://mlsanity.com/paper/1909.13258/) (2019)
- [Background-Click Supervision for Temporal Action Localization](https://mlsanity.com/paper/2111.12449/) (2021)
- [A Weakly Supervised Learning Framework for Salient Object Detection via Hybrid Labels](https://mlsanity.com/paper/2209.02957/) (2022)

## Abstract (arXiv)

Video portrait segmentation (VPS), aiming at segmenting prominent foreground portraits from video frames, has received much attention in recent years. However, simplicity of existing VPS datasets leads to a limitation on extensive research of the task. In this work, we propose a new intricate large-scale Multi-scene Video Portrait Segmentation dataset MVPS consisting of 101 video clips in 7 scenario categories, in which 10,843 sampled frames are finely annotated at pixel level. The dataset has diverse scenes and complicated background environments, which is the most complex dataset in VPS to our best knowledge. Through the observation of a large number of videos with portraits during dataset construction, we find that due to the joint structure of human body, motion of portraits is part-associated, which leads that different parts are relatively independent in motion. That is, motion of different parts of the portraits is imbalanced. Towards this imbalance, an intuitive and reasonable idea is that different motion states in portraits can be better exploited by decoupling the portraits into parts. To achieve this, we propose a Part-Decoupling Network (PDNet) for video portrait segmentation. Specifically, an Inter-frame Part-Discriminated Attention (IPDA) module is proposed which unsupervisedly segments portrait into parts and utilizes different attentiveness on discriminative features specified to each different part. In this way, appropriate attention can be imposed to portrait parts with imbalanced motion to extract part-discriminated correlations, so that the portraits can be segmented more accurately. Experimental results demonstrate that our method achieves leading performance with the comparison to state-of-the-art methods.