# Temporally Coherent Person Matting Trained on Fake-Motion Dataset

arXiv: https://arxiv.org/abs/2109.04843 · HTML: https://mlsanity.com/paper/2109.04843/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [Deep Image Matting](https://mlsanity.com/paper/1703.03872/) (2017)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Background Matting: The World is Your Green Screen](https://mlsanity.com/paper/2004.00626/) (2020)
- [Semantic Human Matting](https://mlsanity.com/paper/1809.01354/) (2018)
- [F, B, Alpha Matting](https://mlsanity.com/paper/2003.07711/) (2020)
- [Boosting Semantic Human Matting with Coarse Annotations](https://mlsanity.com/paper/2004.04955/) (2020)
- [MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1708.05355/) (2017)
- [Towards Real-Time Automatic Portrait Matting on Mobile Devices](https://mlsanity.com/paper/1904.03816/) (2019)
- [Automatic Real-time Background Cut for Portrait Videos](https://mlsanity.com/paper/1704.08812/) (2017)

## Abstract (arXiv)

We propose a novel neural-network-based method to perform matting of videos depicting people that does not require additional user input such as trimaps. Our architecture achieves temporal stability of the resulting alpha mattes by using motion-estimation-based smoothing of image-segmentation algorithm outputs, combined with convolutional-LSTM modules on U-Net skip connections. We also propose a fake-motion algorithm that generates training clips for the video-matting network given photos with ground-truth alpha mattes and background videos. We apply random motion to photos and their mattes to simulate movement one would find in real videos and composite the result with the background clips. It lets us train a deep neural network operating on videos in an absence of a large annotated video dataset and provides ground-truth training-clip foreground optical flow for use in loss functions.