mlsanityData to 7 Oct 2026

Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition

arXiv 2209.14498 ↗First posted 29 Sep 2022cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Deep learning has achieved outstanding performance for face recognition benchmarks, but performance reduces significantly for low resolution (LR) images. We propose an attention similarity knowledge distillation approach, which transfers attention maps obtained from a high resolution (HR) network as a teacher into an LR network as a student to boost LR recognition performance. Inspired by humans being able to approximate an object's region from an LR image based on prior knowledge obtained from HR images, we designed the knowledge distillation loss using the cosine similarity to make the student network's attention resemble the teacher network's attention. Experiments on various LR face related benchmarks confirmed the proposed method generally improved recognition performances on LR settings, outperforming state-of-the-art results by simply transferring well-constructed attention maps. The code and pretrained models are publicly available in the https://github.com/gist-ailab/teaching-where-to-look.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 21 of its 40 references that are in the map. First posted on arXiv 29 Sep 2022.

Builds on

21 of 40 references.

References · 21

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  4. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  5. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  6. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  7. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  8. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  9. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  10. 2019Relational Knowledge Distillationcited by 873
  11. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  12. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 621
9 more
  1. 2014Learning Face Representation from Scratchcited by 451
  2. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  3. 2015WIDER FACE: A Face Detection Benchmarkcited by 297
  4. 2015The MegaFace Benchmark: 1 Million Faces for Recognition at Scalecited by 199
  5. 2018BAM: Bottleneck Attention Modulecited by 171
  6. 2018Low-Resolution Face Recognitioncited by 64
  7. 2020TinaFace: Strong but Simple Baseline for Face Detectioncited by 30
  8. 2019Cross-Resolution Learning for Face Recognitioncited by 22
  9. 2019Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 5; 2026: 4.

22 papers.

Sort:

  1. 2024KeyPoint Relative Position Encoding for Face Recognitioncited by 37
  2. 2024Frequency Attention for Knowledge Distillationcited by 22
  3. 202550 Years of Automated Face Recognitioncited by 13
  4. 2024ARoFace: Alignment Robustness to Improve Low-Quality Face Recognitioncited by 4
  5. 2023One-stage Low-resolution Text Recognition with High-resolution Knowledge Transfercited by 3
  6. 2023Enhancing Low-resolution Face Recognition with Feature Similarity Knowledge Distillationcited by 2
  7. 2023Cross-resolution Face Recognition via Identity-Preserving Network and Knowledge Distillationcited by 2
  8. 2023CCFace: Classification Consistency for Low-Resolution Face Recognitioncited by 1
  9. 2024Boosting Unconstrained Face Recognition with Targeted Style Adversarycited by 1
  10. 2024Knowledge Distillation in RNN-Attention Models for Early Prediction of Student Performancecited by 1
  11. 2025DArFace: Deformation Aware Robustness for Low Quality Face Recognitioncited by 1
  12. 2025UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representationscited by 1
10 more
  1. 2023Deep Boosting Multi-Modal Ensemble Face Recognition with Sample-Level Weightingcited by 0
  2. 2024Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimationcited by 0
  3. 2024Focus on Low-Resolution Information: Multi-Granular Information-Lossless Model for Low-Resolution Human Pose Estimationcited by 0
  4. 2024Low-Resolution Face Recognition via Adaptable Instance-Relation Distillationcited by 0
  5. 2025Unified Knowledge Distillation Framework: Fine-Grained Alignment and Geometric Relationship Preservation for Deep Face Recognitioncited by 0
  6. 2025Global Multiple Extraction Network for Low-Resolution Facial Expression Recognitioncited by 0
  7. 2026Unleashing MLLMs on the Edge: A Unified Framework for Cross-Modal ReID via Adaptive SVD Distillationcited by 0
  8. 2026CD-FKD: Cross-Domain Feature Knowledge Distillation for Robust Single-Domain Generalization in Object Detectioncited by 0
  9. 2026Low-Quality Face Recognition using Center Aligned Representations and Local Margin Constraintscited by 0
  10. 2026Restore What Matters: Lessons from Joint Restoration and Recognitioncited by 0