# Jointly Aligning and Predicting Continuous Emotion Annotations

arXiv: https://arxiv.org/abs/1907.03050 · HTML: https://mlsanity.com/paper/1907.03050/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 4 of 4)

- [Impact of annotation modality on label quality and model performance in the automatic assessment of laughter in-the-wild](https://mlsanity.com/paper/2211.00794/) (2022)
- [Labels Generated by Large Language Models Help Measure People's Empathy in Vitro](https://mlsanity.com/paper/2501.00691/) (2025)
- [Dynamic Time-Alignment of Dimensional Annotations of Emotion using Recurrent Neural Networks](https://mlsanity.com/paper/2209.10223/) (2022)
- [PREFAB: PREFerence-based Affective Modeling for Low-Budget Self-Annotation](https://mlsanity.com/paper/2601.13904/) (2026)

## Abstract (arXiv)

Time-continuous dimensional descriptions of emotions (e.g., arousal, valence) allow researchers to characterize short-time changes and to capture long-term trends in emotion expression. However, continuous emotion labels are generally not synchronized with the input speech signal due to delays caused by reaction-time, which is inherent in human evaluations. To deal with this challenge, we introduce a new convolutional neural network (multi-delay sinc network) that is able to simultaneously align and predict labels in an end-to-end manner. The proposed network is a stack of convolutional layers followed by an aligner network that aligns the speech signal and emotion labels. This network is implemented using a new convolutional layer that we introduce, the delayed sinc layer. It is a time-shifted low-pass (sinc) filter that uses a gradient-based algorithm to learn a single delay. Multiple delayed sinc layers can be used to compensate for a non-stationary delay that is a function of the acoustic space. We test the efficacy of this system on two common emotion datasets, RECOLA and SEWA, and show that this approach obtains state-of-the-art speech-only results by learning time-varying delays while predicting dimensional descriptors of emotions.