# Semi-Supervised Wide-Angle Portraits Correction by Multi-Scale Transformer

arXiv: https://arxiv.org/abs/2109.08024 · HTML: https://mlsanity.com/paper/2109.08024/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Dual Learning for Machine Translation](https://mlsanity.com/paper/1611.00179/) (2016)
- [Learning to Self-Train for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1906.00562/) (2019)
- [ResT: An Efficient Transformer for Visual Recognition](https://mlsanity.com/paper/2105.13677/) (2021)
- [Spatial Uncertainty-Aware Semi-Supervised Crowd Counting](https://mlsanity.com/paper/2107.13271/) (2021)
- [Semi-Supervised Learning for Neural Machine Translation](https://mlsanity.com/paper/1606.04596/) (2016)
- [Semi-Supervised Crowd Counting via Self-Training on Surrogate Tasks](https://mlsanity.com/paper/2007.03207/) (2020)
- [PCC Net: Perspective Crowd Counting via Spatial Convolutional Network](https://mlsanity.com/paper/1905.10085/) (2019)
- [Practical Wide-Angle Portraits Correction with Deep Structured Models](https://mlsanity.com/paper/2104.12464/) (2021)
- [Leveraging Just a Few Keywords for Fine-Grained Aspect Detection Through Weakly Supervised Co-Training](https://mlsanity.com/paper/1909.00415/) (2019)
- [Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks](https://mlsanity.com/paper/1904.04445/) (2019)
- [Transferable Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/1711.06828/) (2017)

## Built on by (most cited first, 10 of 10)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [RecDiffusion: Rectangling for Image Stitching with Diffusion Models](https://mlsanity.com/paper/2403.19164/) (2024)
- [RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning](https://mlsanity.com/paper/2301.01661/) (2023)
- [MOWA: Multiple-in-One Image Warping Model](https://mlsanity.com/paper/2404.10716/) (2024)
- [Semi-Supervised Coupled Thin-Plate Spline Model for Rotation Correction and Beyond](https://mlsanity.com/paper/2401.13432/) (2024)
- [ChrSNet: Chromosome Straightening using Self-attention Guided Networks](https://mlsanity.com/paper/2207.00147/) (2022)
- [Combining Generative and Geometry Priors for Wide-Angle Portrait Correction](https://mlsanity.com/paper/2410.09911/) (2024)
- [Beyond Wide-Angle Images: Structure-to-Detail Video Portrait Correction via Unsupervised Spatiotemporal Adaptation](https://mlsanity.com/paper/2504.00401/) (2025)
- [StableMotion: One-Step Motion Estimation with Diffusion Prior](https://mlsanity.com/paper/2505.06668/) (2025)
- [Rectification Reimagined: A Unified Mamba Model for Image Correction and Rectangling with Prompts](https://mlsanity.com/paper/2512.18718/) (2025)

## Abstract (arXiv)

We propose a semi-supervised network for wide-angle portraits correction. Wide-angle images often suffer from skew and distortion affected by perspective distortion, especially noticeable at the face regions. Previous deep learning based approaches need the ground-truth correction flow maps for training guidance. However, such labels are expensive, which can only be obtained manually. In this work, we design a semi-supervised scheme and build a high-quality unlabeled dataset with rich scenarios, allowing us to simultaneously use labeled and unlabeled data to improve performance. Specifically, our semi-supervised scheme takes advantage of the consistency mechanism, with several novel components such as direction and range consistency (DRC) and regression consistency (RC). Furthermore, different from the existing methods, we propose the Multi-Scale Swin-Unet (MS-Unet) based on the multi-scale swin transformer block (MSTB), which can simultaneously learn short-distance and long-distance information to avoid artifacts. Extensive experiments demonstrate that the proposed method is superior to the state-of-the-art methods and other representative baselines. The source code and dataset are available at: https://github.com/megvii-research/Portraits_Correction.