# ARBEx: Attentive Feature Extraction with Reliability Balancing for Robust Facial Expression Learning

arXiv: https://arxiv.org/abs/2305.01486 · HTML: https://mlsanity.com/paper/2305.01486/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 71 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices](https://mlsanity.com/paper/1804.07573/) (2018)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [A Light CNN for Deep Face Representation with Noisy Labels](https://mlsanity.com/paper/1511.02683/) (2015)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges](https://mlsanity.com/paper/2303.01498/) (2023)
- [Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition](https://mlsanity.com/paper/2104.00232/) (2021)
- [TransFER: Learning Relation-aware Facial Expression Representations with Transformers](https://mlsanity.com/paper/2108.11116/) (2021)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognition](https://mlsanity.com/paper/2207.10299/) (2022)
- [Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition](https://mlsanity.com/paper/2109.07270/) (2021)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition](https://mlsanity.com/paper/2104.05160/) (2021)
- [Transformer-based Multimodal Information Fusion for Facial Expression Analysis](https://mlsanity.com/paper/2203.12367/) (2022)
- [POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition](https://mlsanity.com/paper/2204.04083/) (2022)
- [Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.10335/) (2022)
- [Island Loss for Learning Discriminative Features in Facial Expression Recognition](https://mlsanity.com/paper/1710.03144/) (2017)
- [Expression Snippet Transformer for Robust Video-based Facial Expression Recognition](https://mlsanity.com/paper/2109.08409/) (2021)
- [Towards Semi-Supervised Deep Facial Expression Recognition with An Adaptive Confidence Margin](https://mlsanity.com/paper/2203.12341/) (2022)
- [Uncertainty-aware Label Distribution Learning for Facial Expression Recognition](https://mlsanity.com/paper/2209.10448/) (2022)
- [MVT: Mask Vision Transformer for Facial Expression Recognition in the wild](https://mlsanity.com/paper/2106.04520/) (2021)
- [Face.evoLVe: A High-Performance Face Recognition Library](https://mlsanity.com/paper/2107.08621/) (2021)
- [POSTER++: A simpler and stronger facial expression recognition network](https://mlsanity.com/paper/2301.12149/) (2023)

## Built on by (most cited first, 9 of 9)

- [ResEmoteNet: Bridging Accuracy and Loss Reduction in Facial Emotion Recognition](https://mlsanity.com/paper/2409.10545/) (2024)
- [PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition](https://mlsanity.com/paper/2306.09626/) (2023)
- [Ig3D: Integrating 3D Face Representations in Facial Expression Inference](https://mlsanity.com/paper/2408.16907/) (2024)
- [Versatile audio-visual learning for emotion recognition](https://mlsanity.com/paper/2305.07216/) (2023)
- [Task-adaptive Q-Face](https://mlsanity.com/paper/2405.09059/) (2024)
- [GCF: Graph Convolutional Networks for Facial Expression Recognition](https://mlsanity.com/paper/2407.02361/) (2024)
- [Batch Transformer: Look for Attention in Batch](https://mlsanity.com/paper/2407.04218/) (2024)
- [Facing Asymmetry -- Uncovering the Causal Link between Facial Symmetry and Expression Classifiers using Synthetic Interventions](https://mlsanity.com/paper/2409.15927/) (2024)
- [Sparse Attention to Emotion: Efficient Facial Emotion Recognition via Token Reduction](https://mlsanity.com/paper/2608.08873/) (2026)

## Abstract (arXiv)

In this paper, we introduce a framework ARBEx, a novel attentive feature extraction framework driven by Vision Transformer with reliability balancing to cope against poor class distributions, bias, and uncertainty in the facial expression learning (FEL) task. We reinforce several data pre-processing and refinement methods along with a window-based cross-attention ViT to squeeze the best of the data. We also employ learnable anchor points in the embedding space with label distributions and multi-head self-attention mechanism to optimize performance against weak predictions with reliability balancing, which is a strategy that leverages anchor points, attention scores, and confidence values to enhance the resilience of label predictions. To ensure correct label classification and improve the models' discriminative power, we introduce anchor loss, which encourages large margins between anchor points. Additionally, the multi-head self-attention mechanism, which is also trainable, plays an integral role in identifying accurate labels. This approach provides critical elements for improving the reliability of predictions and has a substantial positive effect on final prediction capabilities. Our adaptive model can be integrated with any deep neural network to forestall challenges in various recognition tasks. Our strategy outperforms current state-of-the-art methodologies, according to extensive experiments conducted in a variety of contexts.