# Estimating the Uncertainty in Emotion Class Labels with Utterance-Specific Dirichlet Priors

arXiv: https://arxiv.org/abs/2203.04443 · HTML: https://mlsanity.com/paper/2203.04443/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 81 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness](https://mlsanity.com/paper/1905.13472/) (2019)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [Emotion Recognition in Speech using Cross-Modal Transfer in the Wild](https://mlsanity.com/paper/1808.05561/) (2018)
- [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](https://mlsanity.com/paper/1806.06228/) (2018)
- [x-vectors meet emotions: A study on dependencies between emotion and speaker recognition](https://mlsanity.com/paper/2002.05039/) (2020)
- [Emotion recognition by fusing time synchronous and time asynchronous representations](https://mlsanity.com/paper/2010.14102/) (2020)
- [Multimodal Sentiment Analysis: Addressing Key Issues and Setting up the Baselines](https://mlsanity.com/paper/1803.07427/) (2018)
- [Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning](https://mlsanity.com/paper/1804.05788/) (2018)
- [Multimodal Emotion Recognition with High-level Speech and Text Features](https://mlsanity.com/paper/2111.10202/) (2021)
- [Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition](https://mlsanity.com/paper/2201.06309/) (2022)
- [CopyPaste: An Augmentation Method for Speech Emotion Recognition](https://mlsanity.com/paper/2010.14602/) (2020)
- [Improved TDNNs using Deep Kernels and Frequency Dependent Grid-RNNs](https://mlsanity.com/paper/1802.06412/) (2018)
- [Speaker diarisation using 2D self-attentive combination of embeddings](https://mlsanity.com/paper/1902.03190/) (2019)

## Built on by (most cited first, 8 of 8)

- [Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression](https://mlsanity.com/paper/2306.06760/) (2023)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [Bayesian WeakS-to-Strong from Text Classification to Generation](https://mlsanity.com/paper/2406.03199/) (2024)
- [Are you sure? Analysing Uncertainty Quantification Approaches for Real-world Speech Emotion Recognition](https://mlsanity.com/paper/2407.01143/) (2024)
- [Distribution-based Emotion Recognition in Conversation](https://mlsanity.com/paper/2211.04834/) (2022)
- [Confidence Estimation for Automatic Detection of Depression and Alzheimer's Disease Based on Clinical Interviews](https://mlsanity.com/paper/2407.19984/) (2024)
- [It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimation](https://mlsanity.com/paper/2310.00486/) (2023)
- [Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition](https://mlsanity.com/paper/2609.05755/) (2026)

## Abstract (arXiv)

Emotion recognition is a key attribute for artificial intelligence systems that need to naturally interact with humans. However, the task definition is still an open problem due to the inherent ambiguity of emotions. In this paper, a novel Bayesian training loss based on per-utterance Dirichlet prior distributions is proposed for verbal emotion recognition, which models the uncertainty in one-hot labels created when human annotators assign the same utterance to different emotion classes. An additional metric is used to evaluate the performance by detection test utterances with high labelling uncertainty. This removes a major limitation that emotion classification systems only consider utterances with labels where the majority of annotators agree on the emotion class. Furthermore, a frequentist approach is studied to leverage the continuous-valued "soft" labels obtained by averaging the one-hot labels. We propose a two-branch model structure for emotion classification on a per-utterance basis, which achieves state-of-the-art classification results on the widely used IEMOCAP dataset. Based on this, uncertainty estimation experiments were performed. The best performance in terms of the area under the precision-recall curve when detecting utterances with high uncertainty was achieved by interpolating the Bayesian training loss with the Kullback-Leibler divergence training loss for the soft labels. The generality of the proposed approach was verified using the MSP-Podcast dataset which yielded the same pattern of results.