# Frequency Aware Face Hallucination Generative Adversarial Network with Semantic Structural Constraint

arXiv: https://arxiv.org/abs/2110.01880 · HTML: https://mlsanity.com/paper/2110.01880/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 27 of 27)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Deeply-Recursive Convolutional Network for Image Super-Resolution](https://mlsanity.com/paper/1511.04491/) (2015)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [Progressive Face Super-Resolution via Attention to Facial Landmark](https://mlsanity.com/paper/1908.08239/) (2019)
- [Learning Spatial Attention for Face Super-Resolution](https://mlsanity.com/paper/2012.01211/) (2020)
- [Super-Identity Convolutional Neural Network for Face Hallucination](https://mlsanity.com/paper/1811.02328/) (2018)
- [Face Super-Resolution Guided by 3D Facial Priors](https://mlsanity.com/paper/2007.09454/) (2020)
- [Super-Resolution with Deep Convolutional Sufficient Statistics](https://mlsanity.com/paper/1511.05666/) (2015)
- [Face Hallucination via Split-Attention in Split-Attention Network](https://mlsanity.com/paper/2010.11575/) (2020)
- [Face hallucination using cascaded super-resolution and identity priors](https://mlsanity.com/paper/1805.10938/) (2018)
- [Component Attention Guided Face Super-Resolution Network: CAGFace](https://mlsanity.com/paper/1910.08761/) (2019)
- [A Frequency Domain Neural Network for Fast Image Super-resolution](https://mlsanity.com/paper/1712.03037/) (2017)
- [Edge and Identity Preserving Network for Face Super-Resolution](https://mlsanity.com/paper/2008.11977/) (2020)
- [DCT-domain Deep Convolutional Neural Networks for Multiple JPEG Compression Classification](https://mlsanity.com/paper/1712.02313/) (2017)
- [Adaptive Transform Domain Image Super-resolution Via Orthogonally Regularized Deep Networks](https://mlsanity.com/paper/1904.10082/) (2019)

## Built on by (most cited first, 1 of 1)

- [Audio-visual video face hallucination with frequency supervision and cross modality support by speech based lip reading loss](https://mlsanity.com/paper/2211.10883/) (2022)

## Abstract (arXiv)

In this paper, we address the issue of face hallucination. Most current face hallucination methods rely on two-dimensional facial priors to generate high resolution face images from low resolution face images. These methods are only capable of assimilating global information into the generated image. Still there exist some inherent problems in these methods; such as, local features, subtle structural details and missing depth information in final output image. Present work proposes a Generative Adversarial Network (GAN) based novel progressive Face Hallucination (FH) network to address these issues present among current methods. The generator of the proposed model comprises of FH network and two sub-networks, assisting FH network to generate high resolution images. The first sub-network leverages on explicitly adding high frequency components into the model. To explicitly encode the high frequency components, an auto encoder is proposed to generate high resolution coefficients of Discrete Cosine Transform (DCT). To add three dimensional parametric information into the network, second sub-network is proposed. This network uses a shape model of 3D Morphable Models (3DMM) to add structural constraint to the FH network. Extensive experimentation results in the paper shows that the proposed model outperforms the state-of-the-art methods.