# Local Learning with Deep and Handcrafted Features for Facial Expression Recognition

arXiv: https://arxiv.org/abs/1804.10892 · HTML: https://mlsanity.com/paper/1804.10892/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2018.

## Built on by (most cited first, 23 of 23)

- [Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition](https://mlsanity.com/paper/2103.09154/) (2021)
- [Facial Emotion Recognition: State of the Art Performance on FER2013](https://mlsanity.com/paper/2105.03588/) (2021)
- [Global-Local Attention for Emotion Recognition](https://mlsanity.com/paper/2111.04129/) (2021)
- [Learning to Amend Facial Expression Representation via De-albino and Affinity](https://mlsanity.com/paper/2103.10189/) (2021)
- [Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition](https://mlsanity.com/paper/2106.03487/) (2021)
- [Disentangling Identity and Pose for Facial Expression Recognition](https://mlsanity.com/paper/2208.08106/) (2022)
- [Teacher-Student Training and Triplet Loss to Reduce the Effect of Drastic Face Occlusion](https://mlsanity.com/paper/2111.10561/) (2021)
- [ResEmoteNet: Bridging Accuracy and Loss Reduction in Facial Emotion Recognition](https://mlsanity.com/paper/2409.10545/) (2024)
- [Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation](https://mlsanity.com/paper/2105.06421/) (2021)
- [EmoNeXt: an Adapted ConvNeXt for Facial Emotion Recognition](https://mlsanity.com/paper/2501.08199/) (2025)
- [Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition](https://mlsanity.com/paper/2109.03336/) (2021)
- [Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training](https://mlsanity.com/paper/2107.05736/) (2021)
- [Hybrid Facial Expression Recognition (FER2013) Model for Real-Time Emotion Classification and Prediction](https://mlsanity.com/paper/2206.09509/) (2022)
- [Dynamic Adaptive Threshold based Learning for Noisy Annotations Robust Facial Expression Recognition](https://mlsanity.com/paper/2208.10221/) (2022)
- [Efficient Neural Architecture Search for Emotion Recognition](https://mlsanity.com/paper/2303.13653/) (2023)
- [Constructing Robust Emotional State-based Feature with a Novel Voting Scheme for Multi-modal Deception Detection in Videos](https://mlsanity.com/paper/2104.08373/) (2021)
- [MAFER: a Multi-resolution Approach to Facial Expression Recognition](https://mlsanity.com/paper/2105.02481/) (2021)
- [Towards a General Deep Feature Extractor for Facial Expression Recognition](https://mlsanity.com/paper/2201.07781/) (2022)
- [Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations](https://mlsanity.com/paper/2107.04746/) (2021)
- [State of the Art: Face Recognition](https://mlsanity.com/paper/2108.11821/) (2021)
- [Multi-Domain Norm-referenced Encoding Enables Data Efficient Transfer Learning of Facial Expression Recognition](https://mlsanity.com/paper/2304.02309/) (2023)
- [Fine-grained 3D object recognition: an approach and experiments](https://mlsanity.com/paper/2306.15919/) (2023)
- [Facial Expression Recognition Using Residual Masking Network](https://mlsanity.com/paper/2603.05937/) (2026)

## Abstract (arXiv)

We present an approach that combines automatic features learned by convolutional neural networks (CNN) and handcrafted features computed by the bag-of-visual-words (BOVW) model in order to achieve state-of-the-art results in facial expression recognition. To obtain automatic features, we experiment with multiple CNN architectures, pre-trained models and training procedures, e.g. Dense-Sparse-Dense. After fusing the two types of features, we employ a local learning framework to predict the class label for each test image. The local learning framework is based on three steps. First, a k-nearest neighbors model is applied in order to select the nearest training samples for an input test image. Second, a one-versus-all Support Vector Machines (SVM) classifier is trained on the selected training samples. Finally, the SVM classifier is used to predict the class label only for the test image it was trained for. Although we have used local learning in combination with handcrafted features in our previous work, to the best of our knowledge, local learning has never been employed in combination with deep features. The experiments on the 2013 Facial Expression Recognition (FER) Challenge data set, the FER+ data set and the AffectNet data set demonstrate that our approach achieves state-of-the-art results. With a top accuracy of 75.42% on FER 2013, 87.76% on the FER+, 59.58% on AffectNet 8-way classification and 63.31% on AffectNet 7-way classification, we surpass the state-of-the-art methods by more than 1% on all data sets.