# Learning Perspective Undistortion of Portraits

arXiv: https://arxiv.org/abs/1905.07515 · HTML: https://mlsanity.com/paper/1905.07515/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2019.

## Built on by (most cited first, 14 of 14)

- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Self-Supervised Camera Self-Calibration from Video](https://mlsanity.com/paper/2112.03325/) (2021)
- [Towards 3D Face Reconstruction in Perspective Projection: Estimating 6DoF Face Pose from Monocular Image](https://mlsanity.com/paper/2205.04126/) (2022)
- [FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold](https://mlsanity.com/paper/2109.09378/) (2021)
- [DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs](https://mlsanity.com/paper/2302.12253/) (2023)
- [Total Selfie: Generating Full-Body Selfies](https://mlsanity.com/paper/2308.14740/) (2023)
- [Correcting Face Distortion in Wide-Angle Videos](https://mlsanity.com/paper/2111.09950/) (2021)
- [Learned Two-Plane Perspective Prior based Image Resampling for Efficient Object Detection](https://mlsanity.com/paper/2303.14311/) (2023)
- [M"obius Transform for Mitigating Perspective Distortions in Representation Learning](https://mlsanity.com/paper/2405.02296/) (2024)
- [SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation](https://mlsanity.com/paper/2406.12700/) (2024)
- [Combining Generative and Geometry Priors for Wide-Angle Portrait Correction](https://mlsanity.com/paper/2410.09911/) (2024)
- [Neural Camera Models](https://mlsanity.com/paper/2208.12903/) (2022)
- [LCM: Log Conformal Maps for Robust Representation Learning to Mitigate Perspective Distortion](https://mlsanity.com/paper/2410.03686/) (2024)
- [An End-to-End Depth-Based Pipeline for Selfie Image Rectification](https://mlsanity.com/paper/2412.19189/) (2024)

## Abstract (arXiv)

Near-range portrait photographs often contain perspective distortion artifacts that bias human perception and challenge both facial recognition and reconstruction techniques. We present the first deep learning based approach to remove such artifacts from unconstrained portraits. In contrast to the previous state-of-the-art approach, our method handles even portraits with extreme perspective distortion, as we avoid the inaccurate and error-prone step of first fitting a 3D face model. Instead, we predict a distortion correction flow map that encodes a per-pixel displacement that removes distortion artifacts when applied to the input image. Our method also automatically infers missing facial features, i.e. occluded ears caused by strong perspective distortion, with coherent details. We demonstrate that our approach significantly outperforms the previous state-of-the-art both qualitatively and quantitatively, particularly for portraits with extreme perspective distortion or facial expressions. We further show that our technique benefits a number of fundamental tasks, significantly improving the accuracy of both face recognition and 3D reconstruction and enables a novel camera calibration technique from a single portrait. Moreover, we also build the first perspective portrait database with a large diversity in identities, expression and poses, which will benefit the related research in this area.