# Mini-ResEmoteNet: Leveraging Knowledge Distillation for Human-Centered Design

arXiv: https://arxiv.org/abs/2501.18538 · HTML: https://mlsanity.com/paper/2501.18538/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond](https://mlsanity.com/paper/2401.01219/) (2024)
- [Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.15123/) (2024)
- [Representation Learning and Identity Adversarial Training for Facial Behavior Understanding](https://mlsanity.com/paper/2407.11243/) (2024)
- [ResEmoteNet: Bridging Accuracy and Loss Reduction in Facial Emotion Recognition](https://mlsanity.com/paper/2409.10545/) (2024)
- [EmoNeXt: an Adapted ConvNeXt for Facial Emotion Recognition](https://mlsanity.com/paper/2501.08199/) (2025)
- [QCS: Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognition](https://mlsanity.com/paper/2411.01988/) (2024)

## Abstract (arXiv)

Facial Emotion Recognition has emerged as increasingly pivotal in the domain of User Experience, notably within modern usability testing, as it facilitates a deeper comprehension of user satisfaction and engagement. This study aims to extend the ResEmoteNet model by employing a knowledge distillation framework to develop Mini-ResEmoteNet models - lightweight student models - tailored for usability testing. Experiments were conducted on the FER2013 and RAF-DB datasets to assess the efficacy of three student model architectures: Student Model A, Student Model B, and Student Model C. Their development involves reducing the number of feature channels in each layer of the teacher model by approximately 50%, 75%, and 87.5%. Demonstrating exceptional performance on the FER2013 dataset, Student Model A (E1) achieved a test accuracy of 76.33%, marking a 0.21% absolute improvement over EmoNeXt. Moreover, the results exhibit absolute improvements in terms of inference speed and memory usage during inference compared to the ResEmoteNet model. The findings indicate that the proposed methods surpass other state-of-the-art approaches.