# Joint Face Image Restoration and Frontalization for Recognition

arXiv: https://arxiv.org/abs/2105.09907 · HTML: https://mlsanity.com/paper/2105.09907/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 76 references that are in the map.
- First posted on arXiv 12 May 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Cross Modal Distillation for Supervision Transfer](https://mlsanity.com/paper/1507.00448/) (2015)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [Deep Semantic Face Deblurring](https://mlsanity.com/paper/1803.03345/) (2018)
- [Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](https://mlsanity.com/paper/1704.04086/) (2017)
- [The Devil of Face Recognition is in the Noise](https://mlsanity.com/paper/1807.11649/) (2018)
- [Examining the Impact of Blur on Recognition by Convolutional Networks](https://mlsanity.com/paper/1611.05760/) (2016)
- [When AWGN-based Denoiser Meets Real Noises](https://mlsanity.com/paper/1904.03485/) (2019)
- [Towards Large-Pose Face Frontalization in the Wild](https://mlsanity.com/paper/1704.06244/) (2017)
- [Super-Identity Convolutional Neural Network for Face Hallucination](https://mlsanity.com/paper/1811.02328/) (2018)
- [Effective Face Frontalization in Unconstrained Images](https://mlsanity.com/paper/1411.7964/) (2014)
- [CR-GAN: Learning Complete Representations for Multi-view Generation](https://mlsanity.com/paper/1806.11191/) (2018)
- [Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing](https://mlsanity.com/paper/1901.05602/) (2019)
- [Copy and Paste GAN: Face Hallucination from Shaded Thumbnails](https://mlsanity.com/paper/2002.10650/) (2020)
- [Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation](https://mlsanity.com/paper/1905.10777/) (2019)
- [Face Hallucination with Finishing Touches](https://mlsanity.com/paper/2002.03308/) (2020)
- [CNN for License Plate Motion Deblurring](https://mlsanity.com/paper/1602.07873/) (2016)
- [Enhance the Motion Cues for Face Anti-Spoofing using CNN-LSTM Architecture](https://mlsanity.com/paper/1901.05635/) (2019)

## Built on by (most cited first, 18 of 18)

- [DifFace: Blind Face Restoration with Diffused Error Contraction](https://mlsanity.com/paper/2212.06512/) (2022)
- [Towards Real-World Blind Face Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2312.15736/) (2023)
- [Survey on Deep Face Restoration: From Non-blind to Blind and Beyond](https://mlsanity.com/paper/2309.15490/) (2023)
- [Deep Face Restoration: A Survey](https://mlsanity.com/paper/2211.02831/) (2022)
- [Underwater Image Restoration Through a Prior Guided Hybrid Sense Approach and Extensive Benchmark Analysis](https://mlsanity.com/paper/2501.02701/) (2025)
- [Image-to-Video Generation via 3D Facial Dynamics](https://mlsanity.com/paper/2105.14678/) (2021)
- [Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoning](https://mlsanity.com/paper/2109.08868/) (2021)
- [Reference-Guided Large-Scale Face Inpainting with Identity and Texture Control](https://mlsanity.com/paper/2303.07014/) (2023)
- [Multi-task Voice Activated Framework using Self-supervised Learning](https://mlsanity.com/paper/2110.01077/) (2021)
- [Information Maximization for Extreme Pose Face Recognition](https://mlsanity.com/paper/2209.03456/) (2022)
- [CATFace: Cross-Attribute-Guided Transformer with Self-Attention Distillation for Low-Quality Face Recognition](https://mlsanity.com/paper/2401.03037/) (2024)
- [Inclusive normalization of face images to passport format](https://mlsanity.com/paper/2312.14544/) (2023)
- [LH2Face: Loss function for Hard High-quality Face](https://mlsanity.com/paper/2506.23555/) (2025)
- [Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior](https://mlsanity.com/paper/2205.14377/) (2022)
- [AS-FIBA: Adaptive Selective Frequency-Injection for Backdoor Attack on Deep Face Restoration](https://mlsanity.com/paper/2403.06430/) (2024)
- [Human Multi-View Synthesis from a Single-View Model:Transferred Body and Face Representations](https://mlsanity.com/paper/2412.03011/) (2024)
- [LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter](https://mlsanity.com/paper/2505.23462/) (2025)
- [Restore What Matters: Lessons from Joint Restoration and Recognition](https://mlsanity.com/paper/2609.13791/) (2026)

## Abstract (arXiv)

In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. To address these challenges, previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. However, most of these methods are stage-wise, which is sub-optimal and deviates from the reality. In this paper, we address all these challenges jointly for unconstrained face recognition. We propose an Multi-Degradation Face Restoration (MDFR) model to restore frontalized high-quality faces from the given low-quality ones under arbitrary facial poses, with three distinct novelties. First, MDFR is a well-designed encoder-decoder architecture which extracts feature representation from an input face image with arbitrary low-quality factors and restores it to a high-quality counterpart. Second, MDFR introduces a pose residual learning strategy along with a 3D-based Pose Normalization Module (PNM), which can perceive the pose gap between the input initial pose and its real-frontal pose to guide the face frontalization. Finally, MDFR can generate frontalized high-quality face images by a single unified network, showing a strong capability of preserving face identity. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks demonstrate the superiority of MDFR over state-of-the-art methods on both face frontalization and face restoration.