# LLV-FSR: Exploiting Large Language-Vision Prior for Face Super-resolution

arXiv: https://arxiv.org/abs/2411.09293 · HTML: https://mlsanity.com/paper/2411.09293/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 14 Nov 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration](https://mlsanity.com/paper/2303.06885/) (2023)
- [GRIT: Faster and Better Image captioning Transformer Using Dual Visual Features](https://mlsanity.com/paper/2207.09666/) (2022)
- [Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation](https://mlsanity.com/paper/2003.13063/) (2020)
- [The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances](https://mlsanity.com/paper/2009.13290/) (2020)
- [Deep Learning-based Face Super-Resolution: A Survey](https://mlsanity.com/paper/2101.03749/) (2021)
- [Towards Open-World Text-Guided Face Image Generation and Manipulation](https://mlsanity.com/paper/2104.08910/) (2021)
- [Face Hallucination via Split-Attention in Split-Attention Network](https://mlsanity.com/paper/2010.11575/) (2020)
- [GAN Prior based Null-Space Learning for Consistent Super-Resolution](https://mlsanity.com/paper/2211.13524/) (2022)
- [WaveFace: Authentic Face Restoration with Efficient Frequency Recovery](https://mlsanity.com/paper/2403.12760/) (2024)
- [CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution](https://mlsanity.com/paper/2204.08696/) (2022)
- [GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond](https://mlsanity.com/paper/2207.14812/) (2022)
- [Efficient Face Super-Resolution via Wavelet-based Feature Enhancement Network](https://mlsanity.com/paper/2407.19768/) (2024)
- [Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration](https://mlsanity.com/paper/2203.08444/) (2022)
- [Semi-Cycled Generative Adversarial Networks for Real-World Face Super-Resolution](https://mlsanity.com/paper/2205.03777/) (2022)
- [A Dive into SAM Prior in Image Restoration](https://mlsanity.com/paper/2305.13620/) (2023)
- [Multi-Prior Learning via Neural Architecture Search for Blind Face Restoration](https://mlsanity.com/paper/2206.13962/) (2022)

## Built on by (most cited first, 2 of 2)

- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [Reference-Guided Identity Preserving Face Restoration](https://mlsanity.com/paper/2505.21905/) (2025)

## Abstract (arXiv)

Existing face super-resolution (FSR) methods have made significant advancements, but they primarily super-resolve face with limited visual information, original pixel-wise space in particular, commonly overlooking the pluralistic clues, like the higher-order depth and semantics, as well as non-visual inputs (text caption and description). Consequently, these methods struggle to produce a unified and meaningful representation from the input face. We suppose that introducing the language-vision pluralistic representation into unexplored potential embedding space could enhance FSR by encoding and exploiting the complementarity across language-vision prior. This motivates us to propose a new framework called LLV-FSR, which marries the power of large vision-language model and higher-order visual prior with the challenging task of FSR. Specifically, besides directly absorbing knowledge from original input, we introduce the pre-trained vision-language model to generate pluralistic priors, involving the image caption, descriptions, face semantic mask and depths. These priors are then employed to guide the more critical feature representation, facilitating realistic and high-quality face super-resolution. Experimental results demonstrate that our proposed framework significantly improves both the reconstruction quality and perceptual quality, surpassing the SOTA by 0.43dB in terms of PSNR on the MMCelebA-HQ dataset.