# Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails

arXiv: https://arxiv.org/abs/2108.00602 · HTML: https://mlsanity.com/paper/2108.00602/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 70 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [Fast Face-swap Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.09577/) (2016)
- [Generative Face Completion](https://mlsanity.com/paper/1704.05838/) (2017)
- [Curriculum Dropout](https://mlsanity.com/paper/1703.06229/) (2017)
- [Pixel Recursive Super Resolution](https://mlsanity.com/paper/1702.00783/) (2017)
- [Deep Cascaded Bi-Network for Face Hallucination](https://mlsanity.com/paper/1607.05046/) (2016)
- [Attention-Aware Face Hallucination via Deep Reinforcement Learning](https://mlsanity.com/paper/1708.03132/) (2017)
- [Geometry-Aware Face Completion and Editing](https://mlsanity.com/paper/1809.02967/) (2018)
- [Learning Oracle Attention for High-fidelity Face Completion](https://mlsanity.com/paper/2003.13903/) (2020)
- [Copy and Paste GAN: Face Hallucination from Shaded Thumbnails](https://mlsanity.com/paper/2002.10650/) (2020)
- [Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition](https://mlsanity.com/paper/2008.11401/) (2020)
- [Face Hallucination with Finishing Touches](https://mlsanity.com/paper/2002.03308/) (2020)
- [Face Hallucination using Linear Models of Coupled Sparse Support](https://mlsanity.com/paper/1512.06009/) (2015)
- [DeMeshNet: Blind Face Inpainting for Deep MeshFace Verification](https://mlsanity.com/paper/1611.05271/) (2016)

## Built on by (most cited first, 4 of 4)

- [CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution](https://mlsanity.com/paper/2204.08696/) (2022)
- [Survey on Deep Face Restoration: From Non-blind to Blind and Beyond](https://mlsanity.com/paper/2309.15490/) (2023)
- [CLIP-SR: Collaborative Linguistic and Image Processing for Super-Resolution](https://mlsanity.com/paper/2412.11609/) (2024)
- [A Unified Framework to Super-Resolve Face Images of Varied Low Resolutions](https://mlsanity.com/paper/2306.03380/) (2023)

## Abstract (arXiv)

In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces (16*16 pixels). Pro-UIGAN iteratively (1) estimates facial geometry priors for low-resolution (LR) faces and (2) acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network super-resolves and inpaints occluded LR faces in a coarse-to-fine manner, thus reducing unwanted blurriness and artifacts significantly. Specifically, we design a novel cross-modal transformer module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutual promotion manner. Extensive experiments demonstrate that our Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks, i.e., face alignment, face parsing, face recognition and expression classification, compared with other state-of-the-art (SotA) methods.