# Towards A Robust Group-level Emotion Recognition via Uncertainty-Aware Learning

arXiv: https://arxiv.org/abs/2310.04306 · HTML: https://mlsanity.com/paper/2310.04306/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 57 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Data Uncertainty Learning in Face Recognition](https://mlsanity.com/paper/2003.11339/) (2020)
- [SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness](https://mlsanity.com/paper/2003.09373/) (2020)
- [Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition](https://mlsanity.com/paper/2104.00232/) (2021)
- [Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification](https://mlsanity.com/paper/2012.08733/) (2020)
- [UCC: Uncertainty guided Cross-head Co-training for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2205.10334/) (2022)
- [Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation](https://mlsanity.com/paper/2203.14098/) (2022)
- [Deep Learning for Human Affect Recognition: Insights and New Developments](https://mlsanity.com/paper/1901.02884/) (2019)
- [Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning](https://mlsanity.com/paper/2112.00568/) (2021)
- [EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction](https://mlsanity.com/paper/1808.07773/) (2018)
- [Non-Volume Preserving-based Fusion to Group-Level Emotion Recognition on Crowd Videos](https://mlsanity.com/paper/1811.11849/) (2018)
- [Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval](https://mlsanity.com/paper/2210.13440/) (2022)
- [Graph Neural Networks for Image Understanding Based on Multiple Cues: Group Emotion Recognition and Event Recognition as Use Cases](https://mlsanity.com/paper/1909.12911/) (2019)
- [An Attention Model for group-level emotion recognition](https://mlsanity.com/paper/1807.03380/) (2018)
- [Group-level Emotion Recognition using Transfer Learning from Face Identification](https://mlsanity.com/paper/1709.01688/) (2017)
- [Emotion Recognition in the Wild using Deep Neural Networks and Bayesian Classifiers](https://mlsanity.com/paper/1709.03820/) (2017)
- [Group Emotion Recognition Using Machine Learning](https://mlsanity.com/paper/1905.01118/) (2019)
- [Group Affect Prediction Using Multimodal Distributions](https://mlsanity.com/paper/1710.01216/) (2017)

## Built on by (most cited first, 4 of 4)

- [A Survey of Deep Learning for Group-level Emotion Recognition](https://mlsanity.com/paper/2408.15276/) (2024)
- [Enhancing Collective Intelligence in Large Language Models Through Emotional Integration](https://mlsanity.com/paper/2503.04849/) (2025)
- [GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos](https://mlsanity.com/paper/2604.16214/) (2026)
- [UAU-Net: Uncertainty-aware Representation Learning and Evidential Classification for Facial Action Unit Detection](https://mlsanity.com/paper/2604.21227/) (2026)

## Abstract (arXiv)

Group-level emotion recognition (GER) is an inseparable part of human behavior analysis, aiming to recognize an overall emotion in a multi-person scene. However, the existing methods are devoted to combing diverse emotion cues while ignoring the inherent uncertainties under unconstrained environments, such as congestion and occlusion occurring within a group. Additionally, since only group-level labels are available, inconsistent emotion predictions among individuals in one group can confuse the network. In this paper, we propose an uncertainty-aware learning (UAL) method to extract more robust representations for GER. By explicitly modeling the uncertainty of each individual, we utilize stochastic embedding drawn from a Gaussian distribution instead of deterministic point embedding. This representation captures the probabilities of different emotions and generates diverse predictions through this stochasticity during the inference stage. Furthermore, uncertainty-sensitive scores are adaptively assigned as the fusion weights of individuals' face within each group. Moreover, we develop an image enhancement module to enhance the model's robustness against severe noise. The overall three-branch model, encompassing face, object, and scene component, is guided by a proportional-weighted fusion strategy and integrates the proposed uncertainty-aware method to produce the final group-level output. Experimental results demonstrate the effectiveness and generalization ability of our method across three widely used databases.