# Towards Large-Pose Face Frontalization in the Wild

arXiv: https://arxiv.org/abs/1704.06244 · HTML: https://mlsanity.com/paper/1704.06244/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2017.

## Built on by (most cited first, 30 of 50)

- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [SynFace: Face Recognition with Synthetic Data](https://mlsanity.com/paper/2108.07960/) (2021)
- [Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images](https://mlsanity.com/paper/2106.07873/) (2021)
- [GINA-3D: Learning to Generate Implicit Neural Assets in the Wild](https://mlsanity.com/paper/2304.02163/) (2023)
- [SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer](https://mlsanity.com/paper/2104.10567/) (2021)
- [Controllable and Guided Face Synthesis for Unconstrained Face Recognition](https://mlsanity.com/paper/2207.10180/) (2022)
- [HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details](https://mlsanity.com/paper/2303.11225/) (2023)
- [Joint Face Image Restoration and Frontalization for Recognition](https://mlsanity.com/paper/2105.09907/) (2021)
- [Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval](https://mlsanity.com/paper/2107.06256/) (2021)
- [LARNet: Lie Algebra Residual Network for Face Recognition](https://mlsanity.com/paper/2103.08147/) (2021)
- [Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis](https://mlsanity.com/paper/2207.10257/) (2022)
- [Egocentric Videoconferencing](https://mlsanity.com/paper/2107.03109/) (2021)
- [BareSkinNet: De-makeup and De-lighting via 3D Face Reconstruction](https://mlsanity.com/paper/2209.09029/) (2022)
- [HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty](https://mlsanity.com/paper/2111.01440/) (2021)
- [SynthDistill: Face Recognition with Knowledge Distillation from Synthetic Data](https://mlsanity.com/paper/2308.14852/) (2023)
- [Unsupervised Disentanglement of Linear-Encoded Facial Semantics](https://mlsanity.com/paper/2103.16605/) (2021)
- [PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition](https://mlsanity.com/paper/2107.11721/) (2021)
- [RPG-Palm: Realistic Pseudo-data Generation for Palmprint Recognition](https://mlsanity.com/paper/2307.14016/) (2023)
- [3D Face Reconstruction: the Road to Forensics](https://mlsanity.com/paper/2309.11357/) (2023)
- [DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs](https://mlsanity.com/paper/2302.12253/) (2023)
- [Heterogeneous Face Frontalization via Domain Agnostic Learning](https://mlsanity.com/paper/2107.08311/) (2021)
- [Expression-preserving face frontalization improves visually assisted speech processing](https://mlsanity.com/paper/2204.02810/) (2022)
- [PAM: Pose Attention Module for Pose-Invariant Face Recognition](https://mlsanity.com/paper/2111.11940/) (2021)
- [Information Maximization for Extreme Pose Face Recognition](https://mlsanity.com/paper/2209.03456/) (2022)
- [Pose Attention-Guided Profile-to-Frontal Face Recognition](https://mlsanity.com/paper/2209.07001/) (2022)
- [ProS: Facial Omni-Representation Learning via Prototype-based Self-Distillation](https://mlsanity.com/paper/2311.01929/) (2023)
- [2D Image head pose estimation via latent space regression under occlusion settings](https://mlsanity.com/paper/2311.06038/) (2023)
- [EyePreserve: Identity-Preserving Iris Synthesis](https://mlsanity.com/paper/2312.12028/) (2023)
- [CATFace: Cross-Attribute-Guided Transformer with Self-Attention Distillation for Low-Quality Face Recognition](https://mlsanity.com/paper/2401.03037/) (2024)
- [UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition](https://mlsanity.com/paper/2502.19803/) (2025)

## Abstract (arXiv)

Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments. Learning pose-invariant features is one solution, but needs expensively labeled large-scale data and carefully designed feature learning algorithms. In this work, we focus on frontalizing faces in the wild under various head poses, including extreme profile views. We propose a novel deep 3D Morphable Model (3DMM) conditioned Face Frontalization Generative Adversarial Network (GAN), termed as FF-GAN, to generate neutral head pose face images. Our framework differs from both traditional GANs and 3DMM based modeling. Incorporating 3DMM into the GAN structure provides shape and appearance priors for fast convergence with less training data, while also supporting end-to-end training. The 3DMM-conditioned GAN employs not only the discriminator and generator loss but also a new masked symmetry loss to retain visual quality under occlusions, besides an identity loss to recover high frequency information. Experiments on face recognition, landmark localization and 3D reconstruction consistently show the advantage of our frontalization method on faces in the wild datasets.