# Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation

arXiv: https://arxiv.org/abs/1811.09998 · HTML: https://mlsanity.com/paper/1811.09998/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2018.

## Built on by (most cited first, 30 of 55)

- [Benchmarking Micro-action Recognition: Dataset, Methods, and Applications](https://mlsanity.com/paper/2403.05234/) (2024)
- [Cross-Quality LFW: A Database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/2108.10290/) (2021)
- [Student Network Learning via Evolutionary Knowledge Distillation](https://mlsanity.com/paper/2103.13811/) (2021)
- [3D Human Pose, Shape and Texture from Low-Resolution Images and Videos](https://mlsanity.com/paper/2103.06498/) (2021)
- [Learning Natural Consistency Representation for Face Forgery Video Detection](https://mlsanity.com/paper/2407.10550/) (2024)
- [Visual Anomaly Detection Via Partition Memory Bank Module and Error Estimation](https://mlsanity.com/paper/2209.12441/) (2022)
- [Recognizability Embedding Enhancement for Very Low-Resolution Face Recognition and Quality Estimation](https://mlsanity.com/paper/2304.10066/) (2023)
- [Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches](https://mlsanity.com/paper/2206.08016/) (2022)
- [Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution](https://mlsanity.com/paper/2109.03075/) (2021)
- [StructVPR: Distill Structural Knowledge with Weighting Samples for Visual Place Recognition](https://mlsanity.com/paper/2212.00937/) (2022)
- [Resolution-Based Distillation for Efficient Histology Image Classification](https://mlsanity.com/paper/2101.04170/) (2021)
- [Deep Rival Penalized Competitive Learning for Low-resolution Face Recognition](https://mlsanity.com/paper/2108.01286/) (2021)
- [Mask-invariant Face Recognition through Template-level Knowledge Distillation](https://mlsanity.com/paper/2112.05646/) (2021)
- [A Comprehensive Review of Knowledge Distillation in Computer Vision](https://mlsanity.com/paper/2404.00936/) (2024)
- [Dual Discriminator Adversarial Distillation for Data-free Model Compression](https://mlsanity.com/paper/2104.05382/) (2021)
- [E3NeRF: Efficient Event-Enhanced Neural Radiance Fields from Blurry Images](https://mlsanity.com/paper/2408.01840/) (2024)
- [Unsupervised domain adaptation for clinician pose estimation and instance segmentation in the operating room](https://mlsanity.com/paper/2108.11801/) (2021)
- [Local-Adaptive Face Recognition via Graph-based Meta-Clustering and Regularized Adaptation](https://mlsanity.com/paper/2203.14327/) (2022)
- [PETALface: Parameter Efficient Transfer Learning for Low-resolution Face Recognition](https://mlsanity.com/paper/2412.07771/) (2024)
- [Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition](https://mlsanity.com/paper/2103.13851/) (2021)
- [Octuplet Loss: Make Face Recognition Robust to Image Resolution](https://mlsanity.com/paper/2207.06726/) (2022)
- [Model Compression Techniques in Biometrics Applications: A Survey](https://mlsanity.com/paper/2401.10139/) (2024)
- [Efficient Low-Resolution Face Recognition via Bridge Distillation](https://mlsanity.com/paper/2409.11786/) (2024)
- [Look One and More: Distilling Hybrid Order Relational Knowledge for Cross-Resolution Image Recognition](https://mlsanity.com/paper/2409.05384/) (2024)
- [Interpretable Embedding Procedure Knowledge Transfer via Stacked Principal Component Analysis and Graph Neural Network](https://mlsanity.com/paper/2104.13561/) (2021)
- [Reasoning Graph Networks for Kinship Verification: from Star-shaped to Hierarchical](https://mlsanity.com/paper/2109.02219/) (2021)
- [CATFace: Cross-Attribute-Guided Transformer with Self-Attention Distillation for Low-Quality Face Recognition](https://mlsanity.com/paper/2401.03037/) (2024)
- [Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation](https://mlsanity.com/paper/2409.02555/) (2024)
- [StructVPR++: Distill Structural and Semantic Knowledge with Weighting Samples for Visual Place Recognition](https://mlsanity.com/paper/2503.06601/) (2025)
- [Dynamic Low-Resolution Distillation for Cost-Efficient End-to-End Text Spotting](https://mlsanity.com/paper/2207.06694/) (2022)

## Abstract (arXiv)

Typically, the deployment of face recognition models in the wild needs to identify low-resolution faces with extremely low computational cost. To address this problem, a feasible solution is compressing a complex face model to achieve higher speed and lower memory at the cost of minimal performance drop. Inspired by that, this paper proposes a learning approach to recognize low-resolution faces via selective knowledge distillation. In this approach, a two-stream convolutional neural network (CNN) is first initialized to recognize high-resolution faces and resolution-degraded faces with a teacher stream and a student stream, respectively. The teacher stream is represented by a complex CNN for high-accuracy recognition, and the student stream is represented by a much simpler CNN for low-complexity recognition. To avoid significant performance drop at the student stream, we then selectively distil the most informative facial features from the teacher stream by solving a sparse graph optimization problem, which are then used to regularize the fine-tuning process of the student stream. In this way, the student stream is actually trained by simultaneously handling two tasks with limited computational resources: approximating the most informative facial cues via feature regression, and recovering the missing facial cues via low-resolution face classification. Experimental results show that the student stream performs impressively in recognizing low-resolution faces and costs only 0.15MB memory and runs at 418 faces per second on CPU and 9,433 faces per second on GPU.