# Privileged Attribution Constrained Deep Networks for Facial Expression Recognition

arXiv: https://arxiv.org/abs/2203.12905 · HTML: https://mlsanity.com/paper/2203.12905/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition](https://mlsanity.com/paper/2104.00232/) (2021)
- [Incorporating Priors with Feature Attribution on Text Classification](https://mlsanity.com/paper/1906.08286/) (2019)
- [JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention](https://mlsanity.com/paper/2003.08834/) (2020)
- [Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition](https://mlsanity.com/paper/2104.05160/) (2021)
- [Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition](https://mlsanity.com/paper/2103.09154/) (2021)
- [Learning Credible Deep Neural Networks with Rationale Regularization](https://mlsanity.com/paper/1908.05601/) (2019)
- [THIN: THrowable Information Networks and Application for Facial Expression Recognition In The Wild](https://mlsanity.com/paper/2010.07614/) (2020)
- [AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition](https://mlsanity.com/paper/2012.14587/) (2020)
- [Tree-gated Deep Mixture-of-Experts For Pose-robust Face Alignment](https://mlsanity.com/paper/1910.09450/) (2019)
- [Confidence-Weighted Local Expression Predictions for Occlusion Handling in Expression Recognition and Action Unit detection](https://mlsanity.com/paper/1607.06290/) (2016)

## Built on by (most cited first, 4 of 4)

- [Guided Interpretable Facial Expression Recognition via Spatial Action Unit Cues](https://mlsanity.com/paper/2402.00281/) (2024)
- [SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance](https://mlsanity.com/paper/2207.04089/) (2022)
- [Spatial Action Unit Cues for Interpretable Deep Facial Expression Recognition](https://mlsanity.com/paper/2410.01848/) (2024)
- [Fighting noise and imbalance in Action Unit detection problems](https://mlsanity.com/paper/2303.02994/) (2023)

## Abstract (arXiv)

Facial Expression Recognition (FER) is crucial in many research domains because it enables machines to better understand human behaviours. FER methods face the problems of relatively small datasets and noisy data that don't allow classical networks to generalize well. To alleviate these issues, we guide the model to concentrate on specific facial areas like the eyes, the mouth or the eyebrows, which we argue are decisive to recognise facial expressions. We propose the Privileged Attribution Loss (PAL), a method that directs the attention of the model towards the most salient facial regions by encouraging its attribution maps to correspond to a heatmap formed by facial landmarks. Furthermore, we introduce several channel strategies that allow the model to have more degrees of freedom. The proposed method is independent of the backbone architecture and doesn't need additional semantic information at test time. Finally, experimental results show that the proposed PAL method outperforms current state-of-the-art methods on both RAF-DB and AffectNet.