# Deep Auto-Encoders with Sequential Learning for Multimodal Dimensional Emotion Recognition

arXiv: https://arxiv.org/abs/2004.13236 · HTML: https://mlsanity.com/paper/2004.13236/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2020.

## Built on by (most cited first, 11 of 11)

- [Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2312.06337/) (2023)
- [A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2203.14779/) (2022)
- [Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition](https://mlsanity.com/paper/2303.08356/) (2023)
- [Cross Attentional Audio-Visual Fusion for Dimensional Emotion Recognition](https://mlsanity.com/paper/2111.05222/) (2021)
- [Masked Graph Learning with Recurrent Alignment for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2407.16714/) (2024)
- [Audio-Visual Fusion for Emotion Recognition in the Valence-Arousal Space Using Joint Cross-Attention](https://mlsanity.com/paper/2209.09068/) (2022)
- [Joint Multimodal Transformer for Emotion Recognition in the Wild](https://mlsanity.com/paper/2403.10488/) (2024)
- [Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild](https://mlsanity.com/paper/2407.12927/) (2024)
- [Two in One Go: Single-stage Emotion Recognition with Decoupled Subject-context Transformer](https://mlsanity.com/paper/2404.17205/) (2024)
- [Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attention](https://mlsanity.com/paper/2407.18552/) (2024)
- [Emotion Embeddings unicodex2014 Learning Stable and Homogeneous Abstractions from Heterogeneous Affective Datasets](https://mlsanity.com/paper/2308.07871/) (2023)

## Abstract (arXiv)

Multimodal dimensional emotion recognition has drawn a great attention from the affective computing community and numerous schemes have been extensively investigated, making a significant progress in this area. However, several questions still remain unanswered for most of existing approaches including: (i) how to simultaneously learn compact yet representative features from multimodal data, (ii) how to effectively capture complementary features from multimodal streams, and (iii) how to perform all the tasks in an end-to-end manner. To address these challenges, in this paper, we propose a novel deep neural network architecture consisting of a two-stream auto-encoder and a long short term memory for effectively integrating visual and audio signal streams for emotion recognition. To validate the robustness of our proposed architecture, we carry out extensive experiments on the multimodal emotion in the wild dataset: RECOLA. Experimental results show that the proposed method achieves state-of-the-art recognition performance and surpasses existing schemes by a significant margin.