# Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition

arXiv: https://arxiv.org/abs/2010.02057 · HTML: https://mlsanity.com/paper/2010.02057/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation](https://mlsanity.com/paper/2207.14087/) (2022)
- [Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusion](https://mlsanity.com/paper/2505.12051/) (2025)
- [Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss](https://mlsanity.com/paper/2207.11482/) (2022)
- [Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism](https://mlsanity.com/paper/2109.15153/) (2021)
- [Exploring Attention Mechanisms for Multimodal Emotion Recognition in an Emergency Call Center Corpus](https://mlsanity.com/paper/2306.07115/) (2023)
- [Gaussian-smoothed Imbalance Data Improves Speech Emotion Recognition](https://mlsanity.com/paper/2302.08650/) (2023)
- [Structures Meet Semantics: Multimodal Fusion via Graph Contrastive Learning](https://mlsanity.com/paper/2508.18322/) (2025)
- [Multi-Microphone and Multi-Modal Emotion Recognition in Reverberant Environment](https://mlsanity.com/paper/2409.09545/) (2024)

## Abstract (arXiv)

This paper aims to bring a new lightweight yet powerful solution for the task of Emotion Recognition and Sentiment Analysis. Our motivation is to propose two architectures based on Transformers and modulation that combine the linguistic and acoustic inputs from a wide range of datasets to challenge, and sometimes surpass, the state-of-the-art in the field. To demonstrate the efficiency of our models, we carefully evaluate their performances on the IEMOCAP, MOSI, MOSEI and MELD dataset. The experiments can be directly replicated and the code is fully open for future researches.