mlsanityData to 14 Oct 2026

Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals

arXiv 2212.13885 ↗First posted 22 Dec 2022eess.SP cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we address the problem of multimodal emotion recognition from multiple physiological signals. We demonstrate that a Transformer-based approach is suitable for this task. In addition, we present how such models may be pretrained in a multimodal scenario to improve emotion recognition performances. We evaluate the benefits of using multimodal inputs and pre-training with our approach on a state-ofthe-art dataset.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 14 of its 36 references that are in the map. First posted on arXiv 22 Dec 2022.

Builds on

14 of 36 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  4. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  5. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  6. 2020Jointly Fine-Tuning "BERT-like" Self Supervised Models to Improve Multimodal Speech Emotion Recognitioncited by 38
  7. 2019Self-supervised Learning for ECG-based Emotion Recognitioncited by 32
  8. 2019Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computingcited by 22
  9. 2022Transformer-Based Self-Supervised Learning for Emotion Recognitioncited by 16
  10. 2020Self-Supervised learning with cross-modal transformers for emotion recognitioncited by 12
  11. 2019A Bayesian Deep Learning Framework for End-To-End Prediction of Emotion from Heartbeatcited by 11
  12. 2021Speech Recognition by Simply Fine-tuning BERTcited by 11
2 more
  1. 2020Unsupervised Multi-Modal Representation Learning for Affective Computing with Multi-Corpus Wearable Datacited by 10
  2. 2017Multi-modal Conditional Attention Fusion for Dimensional Emotion Predictioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 1.

6 papers.

  1. 2024Recent Advances in Multimodal Affective Computing: An NLP Perspectivecited by 13
  2. 2024Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognitioncited by 4
  3. 2025MVP: Multimodal Emotion Recognition based on Video and Physiological Signalscited by 2
  4. 2025Wearable Music2Emotion : Assessing Emotions Induced by AI-Generated Music through Portable EEG-fNIRS Fusioncited by 2
  5. 2023Multi-scale Transformer-based Network for Emotion Recognition from Multi Physiological Signalscited by 1
  6. 2026Deep Temporal Modeling and Ensemble Fusion for Multimodal Emotion Recognition from Physiological Signalscited by 1