# Dynamic Time-Alignment of Dimensional Annotations of Emotion using Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2209.10223 · HTML: https://mlsanity.com/paper/2209.10223/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 34 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge](https://mlsanity.com/paper/1605.01600/) (2016)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [Interpretable Convolutional Filters with SincNet](https://mlsanity.com/paper/1811.09725/) (2018)
- [Capturing Long-term Temporal Dependencies with Convolutional Networks for Continuous Emotion Recognition](https://mlsanity.com/paper/1708.07050/) (2017)
- [Dynamic Difficulty Awareness Training for Continuous Emotion Prediction](https://mlsanity.com/paper/1810.05507/) (2018)
- [Jointly Aligning and Predicting Continuous Emotion Annotations](https://mlsanity.com/paper/1907.03050/) (2019)

## Abstract (arXiv)

Most automatic emotion recognition systems exploit time-continuous annotations of emotion to provide fine-grained descriptions of spontaneous expressions as observed in real-life interactions. As emotion is rather subjective, its annotation is usually performed by several annotators who provide a trace for a given dimension, i.e. a time-continuous series describing a dimension such as arousal or valence. However, annotations of the same expression are rarely consistent between annotators, either in time or in value, which adds bias and delay in the trace that is used to learn predictive models of emotion. We therefore propose a method that can dynamically compensate inconsistencies across annotations and synchronise the traces with the corresponding acoustic features using Recurrent Neural Networks. Experimental evaluations were carried on several emotion data sets that include Chinese, French, German, and Hungarian participants who interacted remotely in either noise-free conditions or in-the-wild. The results show that our method can significantly increase inter-annotator agreement, as well as correlation between traces and audio features, for both arousal and valence. In addition, improvements are obtained in the automatic prediction of these dimensions using simple light-weight models, especially for valence in noise-free conditions, and arousal for recordings captured in-the-wild.