# EFormer: Enhanced Transformer towards Semantic-Contour Features of Foreground for Portraits Matting

arXiv: https://arxiv.org/abs/2308.12831 · HTML: https://mlsanity.com/paper/2308.12831/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [How Do Vision Transformers Work?](https://mlsanity.com/paper/2202.06709/) (2022)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [Deep Image Matting](https://mlsanity.com/paper/1703.03872/) (2017)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://mlsanity.com/paper/2011.11961/) (2020)
- [Background Matting: The World is Your Green Screen](https://mlsanity.com/paper/2004.00626/) (2020)
- [Inception Transformer](https://mlsanity.com/paper/2205.12956/) (2022)
- [U-Net Transformer: Self and Cross Attention for Medical Image Segmentation](https://mlsanity.com/paper/2103.06104/) (2021)
- [Natural Image Matting via Guided Contextual Attention](https://mlsanity.com/paper/2001.04069/) (2020)
- [Mask Transfiner for High-Quality Instance Segmentation](https://mlsanity.com/paper/2111.13673/) (2021)
- [Information-Flow Matting](https://mlsanity.com/paper/1707.05055/) (2017)
- [SeMask: Semantically Masked Transformers for Semantic Segmentation](https://mlsanity.com/paper/2112.12782/) (2021)
- [Deep Video Matting via Spatio-Temporal Alignment and Aggregation](https://mlsanity.com/paper/2104.11208/) (2021)
- [High-Resolution Deep Image Matting](https://mlsanity.com/paper/2009.06613/) (2020)
- [RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer](https://mlsanity.com/paper/2210.07124/) (2022)
- [Fast Deep Matting for Portrait Animation on Mobile Phone](https://mlsanity.com/paper/1707.08289/) (2017)

## Built on by (most cited first, 4 of 4)

- [Efficient Portrait Matte Creation With Layer Diffusion and Connectivity Priors](https://mlsanity.com/paper/2501.16147/) (2025)
- [Capture Stage Matting: Challenges, Approaches, and Solutions for Offline and Real-Time Processing](https://mlsanity.com/paper/2507.07623/) (2025)
- [Learning 3D Texture-Aware Representations for Parsing Diverse Human Clothing and Body Parts](https://mlsanity.com/paper/2508.06032/) (2025)
- [Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AI](https://mlsanity.com/paper/2608.05258/) (2026)

## Abstract (arXiv)

The portrait matting task aims to extract an alpha matte with complete semantics and finely-detailed contours. In comparison to CNN-based approaches, transformers with self-attention module have a better capacity to capture long-range dependencies and low-frequency semantic information of a portrait. However, the recent research shows that self-attention mechanism struggles with modeling high-frequency contour information and capturing fine contour details, which can lead to bias while predicting the portrait's contours. To deal with this issue, we propose EFormer to enhance the model's attention towards both of the low-frequency semantic and high-frequency contour features. For the high-frequency contours, our research demonstrates that cross-attention module between different resolutions can guide our model to allocate attention appropriately to these contour regions. Supported on this, we can successfully extract the high-frequency detail information around the portrait's contours, which are previously ignored by self-attention. Based on cross-attention module, we further build a semantic and contour detector (SCD) to accurately capture both of the low-frequency semantic and high-frequency contour features. And we design contour-edge extraction branch and semantic extraction branch to extract refined high-frequency contour features and complete low-frequency semantic information, respectively. Finally, we fuse the two kinds of features and leverage segmentation head to generate a predicted portrait matte. Experiments on VideoMatte240K (JPEG SD Format) and Adobe Image Matting (AIM) datasets demonstrate that EFormer outperforms previous portrait matte methods.