# Human Instance Matting via Mutual Guidance and Multi-Instance Refinement

arXiv: https://arxiv.org/abs/2205.10767 · HTML: https://mlsanity.com/paper/2205.10767/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 22 May 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [Deep Image Matting](https://mlsanity.com/paper/1703.03872/) (2017)
- [BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation](https://mlsanity.com/paper/2001.00309/) (2020)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [Fully Convolutional Instance-aware Semantic Segmentation](https://mlsanity.com/paper/1611.07709/) (2016)
- [CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinement](https://mlsanity.com/paper/2005.02551/) (2020)
- [Background Matting: The World is Your Green Screen](https://mlsanity.com/paper/2004.00626/) (2020)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [TensorMask: A Foundation for Dense Object Segmentation](https://mlsanity.com/paper/1903.12174/) (2019)
- [Indices Matter: Learning to Index for Deep Image Matting](https://mlsanity.com/paper/1908.00672/) (2019)
- [SSAP: Single-Shot Instance Segmentation With Affinity Pyramid](https://mlsanity.com/paper/1909.01616/) (2019)
- [Mask Guided Matting via Progressive Refinement Network](https://mlsanity.com/paper/2012.06722/) (2020)
- [Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation](https://mlsanity.com/paper/1909.09725/) (2019)
- [Natural Image Matting via Guided Contextual Attention](https://mlsanity.com/paper/2001.04069/) (2020)
- [Semantic Human Matting](https://mlsanity.com/paper/1809.01354/) (2018)
- [Information-Flow Matting](https://mlsanity.com/paper/1707.05055/) (2017)
- [Semantic Image Matting](https://mlsanity.com/paper/2104.08201/) (2021)
- [AlphaGAN: Generative adversarial networks for natural image matting](https://mlsanity.com/paper/1807.10088/) (2018)
- [Boosting Semantic Human Matting with Coarse Annotations](https://mlsanity.com/paper/2004.04955/) (2020)
- [Disentangled Image Matting](https://mlsanity.com/paper/1909.04686/) (2019)

## Built on by (most cited first, 25 of 25)

- [Hi4D: 4D Instance Segmentation of Close Human Interaction](https://mlsanity.com/paper/2303.15380/) (2023)
- [MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation](https://mlsanity.com/paper/2404.02790/) (2024)
- [Matting Anything](https://mlsanity.com/paper/2306.05399/) (2023)
- [Referring Image Matting](https://mlsanity.com/paper/2206.05149/) (2022)
- [MaGGIe: Masked Guided Gradual Human Instance Matting](https://mlsanity.com/paper/2404.16035/) (2024)
- [ZIM: Zero-Shot Image Matting for Anything](https://mlsanity.com/paper/2411.00626/) (2024)
- [Generating Compositional Scenes via Text-to-image RGBA Instance Generation](https://mlsanity.com/paper/2411.10913/) (2024)
- [Robust Human Matting via Semantic Guidance](https://mlsanity.com/paper/2210.05210/) (2022)
- [Deep Image Matting: A Comprehensive Survey](https://mlsanity.com/paper/2304.04672/) (2023)
- [Video Instance Matting](https://mlsanity.com/paper/2311.04212/) (2023)
- [End-to-End Human Instance Matting](https://mlsanity.com/paper/2403.01510/) (2024)
- [In-Context Matting](https://mlsanity.com/paper/2403.15789/) (2024)
- [Towards Natural Image Matting in the Wild via Real-Scenario Prior](https://mlsanity.com/paper/2410.06593/) (2024)
- [Learning from Exemplars for Interactive Image Segmentation](https://mlsanity.com/paper/2406.11472/) (2024)
- [DFIMat: Decoupled Flexible Interactive Matting in Multi-Person Scenarios](https://mlsanity.com/paper/2410.09788/) (2024)
- [Enhancing Image Matting in Real-World Scenes with Mask-Guided Iterative Refinement](https://mlsanity.com/paper/2502.17093/) (2025)
- [MP-Mat: A 3D-and-Instance-Aware Human Matting and Editing Framework with Multiplane Representation](https://mlsanity.com/paper/2504.14606/) (2025)
- [Synthetic Object Compositions for Scalable and Accurate Learning in Detection, Segmentation, and Grounding](https://mlsanity.com/paper/2510.09110/) (2025)
- [Instance-guided Cartoon Editing with a Large-scale Dataset](https://mlsanity.com/paper/2312.01943/) (2023)
- [POBEVM: Real-time Video Matting via Progressively Optimize the Target Body and Edge](https://mlsanity.com/paper/2402.09731/) (2024)
- [DART: Depth-Enhanced Accurate and Real-Time Background Matting](https://mlsanity.com/paper/2402.15820/) (2024)
- [Toward Real-World High-Precision Image Matting and Segmentation](https://mlsanity.com/paper/2601.12080/) (2026)
- [Segment and Matte Anything in a Unified Model](https://mlsanity.com/paper/2601.12147/) (2026)
- [SAM2Matting: Generalized Image and Video Matting](https://mlsanity.com/paper/2606.27339/) (2026)
- [Parallax Portrait Matting](https://mlsanity.com/paper/2607.11205/) (2026)

## Abstract (arXiv)

This paper introduces a new matting task called human instance matting (HIM), which requires the pertinent model to automatically predict a precise alpha matte for each human instance. Straightforward combination of closely related techniques, namely, instance segmentation, soft segmentation and human/conventional matting, will easily fail in complex cases requiring disentangling mingled colors belonging to multiple instances along hairy and thin boundary structures. To tackle these technical challenges, we propose a human instance matting framework, called InstMatt, where a novel mutual guidance strategy working in tandem with a multi-instance refinement module is used, for delineating multi-instance relationship among humans with complex and overlapping boundaries if present. A new instance matting metric called instance matting quality (IMQ) is proposed, which addresses the absence of a unified and fair means of evaluation emphasizing both instance recognition and matting quality. Finally, we construct a HIM benchmark for evaluation, which comprises of both synthetic and natural benchmark images. In addition to thorough experimental results on complex cases with multiple and overlapping human instances each has intricate boundaries, preliminary results are presented on general instance matting. Code and benchmark are available in https://github.com/nowsyn/InstMatt.