# Gated Mechanism for Attention Based Multimodal Sentiment Analysis

arXiv: https://arxiv.org/abs/2003.01043 · HTML: https://mlsanity.com/paper/2003.01043/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2020.

## Built on by (most cited first, 17 of 17)

- [COGMEN: COntextualized GNN based Multimodal Emotion recognitioN](https://mlsanity.com/paper/2205.02455/) (2022)
- [CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection](https://mlsanity.com/paper/2204.05515/) (2022)
- [Defending Multimodal Fusion Models against Single-Source Adversaries](https://mlsanity.com/paper/2206.12714/) (2022)
- [MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2201.09828/) (2022)
- [What BERT Based Language Models Learn in Spoken Transcripts: An Empirical Study](https://mlsanity.com/paper/2109.09105/) (2021)
- [MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models](https://mlsanity.com/paper/2403.19080/) (2024)
- [Multimodal Vision Transformers with Forced Attention for Behavior Analysis](https://mlsanity.com/paper/2212.03968/) (2022)
- [Cross-Attention is Not Always Needed: Dynamic Cross-Attention for Audio-Visual Dimensional Emotion Recognition](https://mlsanity.com/paper/2403.19554/) (2024)
- [Multi-channel Attentive Graph Convolutional Network With Sentiment Fusion For Multimodal Sentiment Analysis](https://mlsanity.com/paper/2201.10274/) (2022)
- [SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognition](https://mlsanity.com/paper/2407.20920/) (2024)
- [Bridging Modalities: Knowledge Distillation and Masked Training for Translating Multi-Modal Emotion Recognition to Uni-Modal, Speech-Only Emotion Recognition](https://mlsanity.com/paper/2401.03000/) (2024)
- [Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2405.12853/) (2024)
- [CorMulT: A Semi-supervised Modality Correlation-aware Multimodal Transformer for Sentiment Analysis](https://mlsanity.com/paper/2407.07046/) (2024)
- [Structures Meet Semantics: Multimodal Fusion via Graph Contrastive Learning](https://mlsanity.com/paper/2508.18322/) (2025)
- [Low Resource Pipeline for Spoken Language Understanding via Weak Supervision](https://mlsanity.com/paper/2206.10559/) (2022)
- [CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets](https://mlsanity.com/paper/2501.03332/) (2025)
- [United we stand, Divided we fall: Handling Weak Complementary Relationships for Audio-Visual Emotion Recognition in Valence-Arousal Space](https://mlsanity.com/paper/2503.12261/) (2025)

## Abstract (arXiv)

Multimodal sentiment analysis has recently gained popularity because of its relevance to social media posts, customer service calls and video blogs. In this paper, we address three aspects of multimodal sentiment analysis; 1. Cross modal interaction learning, i.e. how multiple modalities contribute to the sentiment, 2. Learning long-term dependencies in multimodal interactions and 3. Fusion of unimodal and cross modal cues. Out of these three, we find that learning cross modal interactions is beneficial for this problem. We perform experiments on two benchmark datasets, CMU Multimodal Opinion level Sentiment Intensity (CMU-MOSI) and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus. Our approach on both these tasks yields accuracies of 83.9% and 81.1% respectively, which is 1.6% and 1.34% absolute improvement over current state-of-the-art.