# Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition

arXiv: https://arxiv.org/abs/2209.14498 · HTML: https://mlsanity.com/paper/2209.14498/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 21 of 21)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [Low-Resolution Face Recognition](https://mlsanity.com/paper/1811.08965/) (2018)
- [TinaFace: Strong but Simple Baseline for Face Detection](https://mlsanity.com/paper/2011.13183/) (2020)
- [Cross-Resolution Learning for Face Recognition](https://mlsanity.com/paper/1912.02851/) (2019)
- [Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation](https://mlsanity.com/paper/1905.10777/) (2019)

## Built on by (most cited first, 22 of 22)

- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [Frequency Attention for Knowledge Distillation](https://mlsanity.com/paper/2403.05894/) (2024)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [ARoFace: Alignment Robustness to Improve Low-Quality Face Recognition](https://mlsanity.com/paper/2407.14972/) (2024)
- [One-stage Low-resolution Text Recognition with High-resolution Knowledge Transfer](https://mlsanity.com/paper/2308.02770/) (2023)
- [Enhancing Low-resolution Face Recognition with Feature Similarity Knowledge Distillation](https://mlsanity.com/paper/2303.04681/) (2023)
- [Cross-resolution Face Recognition via Identity-Preserving Network and Knowledge Distillation](https://mlsanity.com/paper/2303.08665/) (2023)
- [CCFace: Classification Consistency for Low-Resolution Face Recognition](https://mlsanity.com/paper/2308.09230/) (2023)
- [Boosting Unconstrained Face Recognition with Targeted Style Adversary](https://mlsanity.com/paper/2408.07642/) (2024)
- [Knowledge Distillation in RNN-Attention Models for Early Prediction of Student Performance](https://mlsanity.com/paper/2412.14526/) (2024)
- [DArFace: Deformation Aware Robustness for Low Quality Face Recognition](https://mlsanity.com/paper/2505.08423/) (2025)
- [UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representations](https://mlsanity.com/paper/2510.24116/) (2025)
- [Deep Boosting Multi-Modal Ensemble Face Recognition with Sample-Level Weighting](https://mlsanity.com/paper/2308.09234/) (2023)
- [Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation](https://mlsanity.com/paper/2405.11448/) (2024)
- [Focus on Low-Resolution Information: Multi-Granular Information-Lossless Model for Low-Resolution Human Pose Estimation](https://mlsanity.com/paper/2405.12247/) (2024)
- [Low-Resolution Face Recognition via Adaptable Instance-Relation Distillation](https://mlsanity.com/paper/2409.02049/) (2024)
- [Unified Knowledge Distillation Framework: Fine-Grained Alignment and Geometric Relationship Preservation for Deep Face Recognition](https://mlsanity.com/paper/2508.11376/) (2025)
- [Global Multiple Extraction Network for Low-Resolution Facial Expression Recognition](https://mlsanity.com/paper/2511.05938/) (2025)
- [Unleashing MLLMs on the Edge: A Unified Framework for Cross-Modal ReID via Adaptive SVD Distillation](https://mlsanity.com/paper/2602.12936/) (2026)
- [CD-FKD: Cross-Domain Feature Knowledge Distillation for Robust Single-Domain Generalization in Object Detection](https://mlsanity.com/paper/2603.16439/) (2026)
- [Low-Quality Face Recognition using Center Aligned Representations and Local Margin Constraints](https://mlsanity.com/paper/2609.01014/) (2026)
- [Restore What Matters: Lessons from Joint Restoration and Recognition](https://mlsanity.com/paper/2609.13791/) (2026)

## Abstract (arXiv)

Deep learning has achieved outstanding performance for face recognition benchmarks, but performance reduces significantly for low resolution (LR) images. We propose an attention similarity knowledge distillation approach, which transfers attention maps obtained from a high resolution (HR) network as a teacher into an LR network as a student to boost LR recognition performance. Inspired by humans being able to approximate an object's region from an LR image based on prior knowledge obtained from HR images, we designed the knowledge distillation loss using the cosine similarity to make the student network's attention resemble the teacher network's attention. Experiments on various LR face related benchmarks confirmed the proposed method generally improved recognition performances on LR settings, outperforming state-of-the-art results by simply transferring well-constructed attention maps. The code and pretrained models are publicly available in the https://github.com/gist-ailab/teaching-where-to-look.