# Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition

arXiv: https://arxiv.org/abs/2109.03336 · HTML: https://mlsanity.com/paper/2109.03336/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 67 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning](https://mlsanity.com/paper/1803.04765/) (2018)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network](https://mlsanity.com/paper/1910.11111/) (2019)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [Deep k-NN for Noisy Labels](https://mlsanity.com/paper/2004.12289/) (2020)
- [Going Deeper in Facial Expression Recognition using Deep Neural Networks](https://mlsanity.com/paper/1511.04110/) (2015)
- [Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network](https://mlsanity.com/paper/1902.01019/) (2019)
- [Local Learning with Deep and Handcrafted Features for Facial Expression Recognition](https://mlsanity.com/paper/1804.10892/) (2018)
- [Peak-Piloted Deep Network for Facial Expression Recognition](https://mlsanity.com/paper/1607.06997/) (2016)
- [Facial Motion Prior Networks for Facial Expression Recognition](https://mlsanity.com/paper/1902.08788/) (2019)
- [Learning to Amend Facial Expression Representation via De-albino and Affinity](https://mlsanity.com/paper/2103.10189/) (2021)
- [Landmark Guidance Independent Spatio-channel Attention and Complementary Context Information based Facial Expression Recognition](https://mlsanity.com/paper/2007.10298/) (2020)
- [A review of learning vector quantization classifiers](https://mlsanity.com/paper/1509.07093/) (2015)
- [Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines](https://mlsanity.com/paper/1604.03225/) (2016)
- [Deep-RBF Networks Revisited: Robust Classification with Rejection](https://mlsanity.com/paper/1812.03190/) (2018)
- [A Real Time Facial Expression Classification System Using Local Binary Patterns](https://mlsanity.com/paper/1505.04058/) (2015)
- [Recognition of facial expressions based on salient geometric features and support vector machines](https://mlsanity.com/paper/1604.04334/) (2016)
- [Multimodal Local-Global Ranking Fusion for Emotion Recognition](https://mlsanity.com/paper/1809.04931/) (2018)

## Built on by (most cited first, 3 of 3)

- [Defensive Patches for Robust Recognition in the Physical World](https://mlsanity.com/paper/2204.06213/) (2022)
- [A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognition](https://mlsanity.com/paper/2111.02172/) (2021)
- [Affective Video Content Analysis: Decade Review and New Perspectives](https://mlsanity.com/paper/2310.17212/) (2023)

## Abstract (arXiv)

Emotion recognition (ER) from facial images is one of the landmark tasks in affective computing with major developments in the last decade. Initial efforts on ER relied on handcrafted features that were used to characterize facial images and then feed to standard predictive models. Recent methodologies comprise end-to-end trainable deep learning methods that simultaneously learn both, features and predictive model. Perhaps the most successful models are based on convolutional neural networks (CNNs). While these models have excelled at this task, they still fail at capturing local patterns that could emerge in the learning process. We hypothesize these patterns could be captured by variants based on locally weighted learning. Specifically, in this paper we propose a CNN based architecture enhanced with multiple branches formed by radial basis function (RBF) units that aims at exploiting local information at the final stage of the learning process. Intuitively, these RBF units capture local patterns shared by similar instances using an intermediate representation, then the outputs of the RBFs are feed to a softmax layer that exploits this information to improve the predictive performance of the model. This feature could be particularly advantageous in ER as cultural / ethnicity differences may be identified by the local units. We evaluate the proposed method in several ER datasets and show the proposed methodology achieves state-of-the-art in some of them, even when we adopt a pre-trained VGG-Face model as backbone. We show it is the incorporation of local information what makes the proposed model competitive.