# MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition

arXiv: https://arxiv.org/abs/2011.11961 · HTML: https://mlsanity.com/paper/2011.11961/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 30 of 127)

- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [MegaPortraits: One-shot Megapixel Neural Head Avatars](https://mlsanity.com/paper/2207.07621/) (2022)
- [Fusing Global and Local Features for Generalized AI-Synthesized Image Detection](https://mlsanity.com/paper/2203.13964/) (2022)
- [KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints](https://mlsanity.com/paper/2205.04992/) (2022)
- [What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?](https://mlsanity.com/paper/2403.06090/) (2024)
- [Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation](https://mlsanity.com/paper/2209.10510/) (2022)
- [Deep Gradient Learning for Efficient Camouflaged Object Detection](https://mlsanity.com/paper/2205.12853/) (2022)
- [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022)
- [GGHead: Fast and Generalizable 3D Gaussian Heads](https://mlsanity.com/paper/2406.09377/) (2024)
- [LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar](https://mlsanity.com/paper/2305.01190/) (2023)
- [GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation](https://mlsanity.com/paper/2312.01632/) (2023)
- [Learning Disentangled Avatars with Hybrid 3D Representations](https://mlsanity.com/paper/2309.06441/) (2023)
- [Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction](https://mlsanity.com/paper/2306.05872/) (2023)
- [TriPlaneNet: An Encoder for EG3D Inversion](https://mlsanity.com/paper/2303.13497/) (2023)
- [FLARE: Fast Learning of Animatable and Relightable Mesh Avatars](https://mlsanity.com/paper/2310.17519/) (2023)
- [MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation](https://mlsanity.com/paper/2404.02790/) (2024)
- [Generative Omnimatte: Learning to Decompose Video into Layers](https://mlsanity.com/paper/2411.16683/) (2024)
- [Text2Layer: Layered Image Generation using Latent Diffusion Model](https://mlsanity.com/paper/2307.09781/) (2023)
- [MonoNPHM: Dynamic Head Reconstruction from Monocular Videos](https://mlsanity.com/paper/2312.06740/) (2023)
- [Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image](https://mlsanity.com/paper/2403.09632/) (2024)
- [FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video](https://mlsanity.com/paper/2411.15604/) (2024)
- [Matting Anything](https://mlsanity.com/paper/2306.05399/) (2023)
- [GLFF: Global and Local Feature Fusion for AI-synthesized Image Detection](https://mlsanity.com/paper/2211.08615/) (2022)
- [MonoHair: High-Fidelity Hair Modeling from a Monocular Video](https://mlsanity.com/paper/2403.18356/) (2024)
- [MatAnyone: Stable Video Matting with Consistent Memory Propagation](https://mlsanity.com/paper/2501.14677/) (2025)
- [Referring Image Matting](https://mlsanity.com/paper/2206.05149/) (2022)
- [Generalizable One-shot Neural Head Avatar](https://mlsanity.com/paper/2306.08768/) (2023)
- [Real-time 3D-aware Portrait Video Relighting](https://mlsanity.com/paper/2410.18355/) (2024)

## Abstract (arXiv)

Existing portrait matting methods either require auxiliary inputs that are costly to obtain or involve multiple stages that are computationally expensive, making them less suitable for real-time applications. In this work, we present a light-weight matting objective decomposition network (MODNet) for portrait matting in real-time with a single input image. The key idea behind our efficient design is by optimizing a series of sub-objectives simultaneously via explicit constraints. In addition, MODNet includes two novel techniques for improving model efficiency and robustness. First, an Efficient Atrous Spatial Pyramid Pooling (e-ASPP) module is introduced to fuse multi-scale features for semantic estimation. Second, a self-supervised sub-objectives consistency (SOC) strategy is proposed to adapt MODNet to real-world data to address the domain shift problem common to trimap-free methods. MODNet is easy to be trained in an end-to-end manner. It is much faster than contemporaneous methods and runs at 67 frames per second on a 1080Ti GPU. Experiments show that MODNet outperforms prior trimap-free methods by a large margin on both Adobe Matting Dataset and a carefully designed photographic portrait matting (PPM-100) benchmark proposed by us. Further, MODNet achieves remarkable results on daily photos and videos. Our code and models are available at https://github.com/ZHKKKe/MODNet, and the PPM-100 benchmark is released at https://github.com/ZHKKKe/PPM.