# Occlusion Aware Student Emotion Recognition based on Facial Action Unit Detection

arXiv: https://arxiv.org/abs/2307.09465 · HTML: https://mlsanity.com/paper/2307.09465/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 47 references that are in the map.
- First posted on arXiv 18 Jul 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition](https://mlsanity.com/paper/2104.00232/) (2021)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/1904.09939/) (2019)
- [Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition](https://mlsanity.com/paper/2104.05160/) (2021)
- [Masked Face Recognition for Secure Authentication](https://mlsanity.com/paper/2008.11104/) (2020)
- [Deformable Part Models are Convolutional Neural Networks](https://mlsanity.com/paper/1409.5403/) (2014)
- [Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution](https://mlsanity.com/paper/2004.09681/) (2020)
- [Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition](https://mlsanity.com/paper/2103.13372/) (2021)
- [EmotioNet Challenge: Recognition of facial expressions of emotion in the wild](https://mlsanity.com/paper/1703.01210/) (2017)
- [Hand2Face: Automatic Synthesis and Recognition of Hand Over Face Occlusions](https://mlsanity.com/paper/1708.00370/) (2017)

## Abstract (arXiv)

Given that approximately half of science, technology, engineering, and mathematics (STEM) undergraduate students in U.S. colleges and universities leave by the end of the first year [15], it is crucial to improve the quality of classroom environments. This study focuses on monitoring students' emotions in the classroom as an indicator of their engagement and proposes an approach to address this issue. The impact of different facial parts on the performance of an emotional recognition model is evaluated through experimentation. To test the proposed model under partial occlusion, an artificially occluded dataset is introduced. The novelty of this work lies in the proposal of an occlusion-aware architecture for facial action units (AUs) extraction, which employs attention mechanism and adaptive feature learning. The AUs can be used later to classify facial expressions in classroom settings. This research paper's findings provide valuable insights into handling occlusion in analyzing facial images for emotional engagement analysis. The proposed experiments demonstrate the significance of considering occlusion and enhancing the reliability of facial analysis models in classroom environments. These findings can also be extended to other settings where occlusions are prevalent.