# Distilling Generative-Discriminative Representations for Very Low-Resolution Face Recognition

arXiv: https://arxiv.org/abs/2409.06371 · HTML: https://mlsanity.com/paper/2409.06371/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Diffusion Model as Representation Learner](https://mlsanity.com/paper/2308.10916/) (2023)
- [Low-Resolution Face Recognition](https://mlsanity.com/paper/1811.08965/) (2018)
- [Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation](https://mlsanity.com/paper/1811.09998/) (2018)
- [PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance](https://mlsanity.com/paper/2309.10810/) (2023)
- [VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition](https://mlsanity.com/paper/1910.04985/) (2019)
- [Student Network Learning via Evolutionary Knowledge Distillation](https://mlsanity.com/paper/2103.13811/) (2021)
- [Dual Directed Capsule Network for Very Low Resolution Image Recognition](https://mlsanity.com/paper/1908.10027/) (2019)
- [Unconstrained Face Detection and Open-Set Face Recognition Challenge](https://mlsanity.com/paper/1708.02337/) (2017)
- [Recognizability Embedding Enhancement for Very Low-Resolution Face Recognition and Quality Estimation](https://mlsanity.com/paper/2304.10066/) (2023)
- [Deep Rival Penalized Competitive Learning for Low-resolution Face Recognition](https://mlsanity.com/paper/2108.01286/) (2021)
- [Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation](https://mlsanity.com/paper/2409.02555/) (2024)
- [Cross-resolution Face Recognition via Identity-Preserving Network and Knowledge Distillation](https://mlsanity.com/paper/2303.08665/) (2023)
- [Masked Face Recognition with Generative-to-Discriminative Representations](https://mlsanity.com/paper/2405.16761/) (2024)
- [CCFace: Classification Consistency for Low-Resolution Face Recognition](https://mlsanity.com/paper/2308.09230/) (2023)

## Abstract (arXiv)

Very low-resolution face recognition is challenging due to the serious loss of informative facial details in resolution degradation. In this paper, we propose a generative-discriminative representation distillation approach that combines generative representation with cross-resolution aligned knowledge distillation. This approach facilitates very low-resolution face recognition by jointly distilling generative and discriminative models via two distillation modules. Firstly, the generative representation distillation takes the encoder of a diffusion model pretrained for face super-resolution as the generative teacher to supervise the learning of the student backbone via feature regression, and then freezes the student backbone. After that, the discriminative representation distillation further considers a pretrained face recognizer as the discriminative teacher to supervise the learning of the student head via cross-resolution relational contrastive distillation. In this way, the general backbone representation can be transformed into discriminative head representation, leading to a robust and discriminative student model for very low-resolution face recognition. Our approach improves the recovery of the missing details in very low-resolution faces and achieves better knowledge transfer. Extensive experiments on face datasets demonstrate that our approach enhances the recognition accuracy of very low-resolution faces, showcasing its effectiveness and adaptability.