# ShrinkTeaNet: Million-scale Lightweight Face Recognition via Shrinking Teacher-Student Networks

arXiv: https://arxiv.org/abs/1905.10620 · HTML: https://mlsanity.com/paper/1905.10620/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2019.

## Built on by (most cited first, 12 of 12)

- [The Right to Talk: An Audio-Visual Transformer Approach](https://mlsanity.com/paper/2108.03256/) (2021)
- [BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in Semantic Scene Segmentation](https://mlsanity.com/paper/2108.03267/) (2021)
- [Teacher-Student Architecture for Knowledge Distillation: A Survey](https://mlsanity.com/paper/2308.04268/) (2023)
- [Teacher-Student Architecture for Knowledge Learning: A Survey](https://mlsanity.com/paper/2210.17332/) (2022)
- [Evaluation-oriented Knowledge Distillation for Deep Face Recognition](https://mlsanity.com/paper/2206.02325/) (2022)
- [AdaDistill: Adaptive Knowledge Distillation for Deep Face Recognition](https://mlsanity.com/paper/2407.01332/) (2024)
- [Grouped Knowledge Distillation for Deep Face Recognition](https://mlsanity.com/paper/2304.04462/) (2023)
- [Model Compression Techniques in Biometrics Applications: A Survey](https://mlsanity.com/paper/2401.10139/) (2024)
- [Prototype Memory for Large-scale Face Representation Learning](https://mlsanity.com/paper/2105.02103/) (2021)
- [Rethinking Position Bias Modeling with Knowledge Distillation for CTR Prediction](https://mlsanity.com/paper/2204.00270/) (2022)
- [Joint Feature Distribution Alignment Learning for NIR-VIS and VIS-VIS Face Recognition](https://mlsanity.com/paper/2204.11434/) (2022)
- [Unified Knowledge Distillation Framework: Fine-Grained Alignment and Geometric Relationship Preservation for Deep Face Recognition](https://mlsanity.com/paper/2508.11376/) (2025)

## Abstract (arXiv)

Large-scale face recognition in-the-wild has been recently achieved matured performance in many real work applications. However, such systems are built on GPU platforms and mostly deploy heavy deep network architectures. Given a high-performance heavy network as a teacher, this work presents a simple and elegant teacher-student learning paradigm, namely ShrinkTeaNet, to train a portable student network that has significantly fewer parameters and competitive accuracy against the teacher network. Far apart from prior teacher-student frameworks mainly focusing on accuracy and compression ratios in closed-set problems, our proposed teacher-student network is proved to be more robust against open-set problem, i.e. large-scale face recognition. In addition, this work introduces a novel Angular Distillation Loss for distilling the feature direction and the sample distributions of the teacher's hypersphere to its student. Then ShrinkTeaNet framework can efficiently guide the student's learning process with the teacher's knowledge presented in both intermediate and last stages of the feature embedding. Evaluations on LFW, CFP-FP, AgeDB, IJB-B and IJB-C Janus, and MegaFace with one million distractors have demonstrated the efficiency of the proposed approach to learn robust student networks which have satisfying accuracy and compact sizes. Our ShrinkTeaNet is able to support the light-weight architecture achieving high performance with 99.77% on LFW and 95.64% on large-scale Megaface protocols.