# Beyond the Mean: Modelling Annotation Distributions in Continuous Affect Prediction

arXiv: https://arxiv.org/abs/2604.07198 · HTML: https://mlsanity.com/paper/2604.07198/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network](https://mlsanity.com/paper/2103.00110/) (2021)
- [From the Lab to the Wild: Affect Modeling via Privileged Information](https://mlsanity.com/paper/2305.10919/) (2023)
- [Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression](https://mlsanity.com/paper/2306.06760/) (2023)
- [End-To-End Label Uncertainty Modeling for Speech-based Arousal Recognition Using Bayesian Neural Networks](https://mlsanity.com/paper/2110.03299/) (2021)
- [Predicting Player Engagement in Tom Clancy's The Division 2: A Multimodal Approach via Pixels and Gamepad Actions](https://mlsanity.com/paper/2310.06136/) (2023)
- [Supervised Contrastive Learning for Affect Modelling](https://mlsanity.com/paper/2208.12238/) (2022)
- [The Invariant Ground Truth of Affect](https://mlsanity.com/paper/2210.07630/) (2022)
- [Conformalized Multimodal Uncertainty Regression and Reasoning](https://mlsanity.com/paper/2309.11018/) (2023)
- [Across-Game Engagement Modelling via Few-Shot Learning](https://mlsanity.com/paper/2409.13002/) (2024)
- [Privileged Contrastive Pretraining for Multimodal Affect Modelling](https://mlsanity.com/paper/2508.03729/) (2025)
- [Predicting score distribution to improve non-intrusive speech quality estimation](https://mlsanity.com/paper/2204.06616/) (2022)
- [Beta quantile regression for robust estimation of uncertainty in the presence of outliers](https://mlsanity.com/paper/2309.07374/) (2023)
- [Emotions as Ambiguity-aware Ordinal Representations](https://mlsanity.com/paper/2508.19193/) (2025)

## Abstract (arXiv)

Emotion annotation is inherently subjective and cognitively demanding, producing signals that reflect diverse perceptions across annotators rather than a single ground truth. In continuous affect prediction, this variability is typically collapsed into point estimates such as the mean or median, discarding valuable information about annotator disagreement and uncertainty. In this work, we propose a distribution-aware framework that models annotation consensus using the Beta distribution. Instead of predicting a single affect value, models estimate the mean and standard deviation of the annotation distribution, which are transformed into valid Beta parameters through moment matching. This formulation enables the recovery of higher-order distributional descriptors, including skewness, kurtosis, and quantiles, in closed form. As a result, the model captures not only the central tendency of emotional perception but also variability, asymmetry, and uncertainty in annotator responses. We evaluate the proposed approach on the SEWA and RECOLA datasets using multimodal features. Experimental results show that Beta-based modelling produces predictive distributions that closely match the empirical annotator distributions while achieving competitive performance with conventional regression approaches. These findings highlight the importance of modelling annotation uncertainty in affective computing and demonstrate the potential of distribution-aware learning for subjective signal analysis.