# Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis

arXiv: https://arxiv.org/abs/1904.08138 · HTML: https://mlsanity.com/paper/1904.08138/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 11 of 11)

- [Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis](https://mlsanity.com/paper/2107.13669/) (2021)
- [Multimodal Learning using Optimal Transport for Sarcasm and Humor Detection](https://mlsanity.com/paper/2110.10949/) (2021)
- [Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition](https://mlsanity.com/paper/2308.04502/) (2023)
- [Multimodal Sentiment Analysis: A Survey](https://mlsanity.com/paper/2305.07611/) (2023)
- [Distributed Federated Learning-Based Deep Learning Model for Privacy MRI Brain Tumor Detection](https://mlsanity.com/paper/2404.10026/) (2024)
- [Multistage linguistic conditioning of convolutional layers for speech emotion recognition](https://mlsanity.com/paper/2110.06650/) (2021)
- [Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion with Missing Data](https://mlsanity.com/paper/2210.17311/) (2022)
- [Machine learning-based system reliability analysis with Gaussian Process Regression](https://mlsanity.com/paper/2403.11125/) (2024)
- [Cross-Modal Domain Adaptation in Brain Disease Diagnosis: Maximum Mean Discrepancy-based Convolutional Neural Networks](https://mlsanity.com/paper/2405.03235/) (2024)
- [Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers](https://mlsanity.com/paper/2306.13804/) (2023)
- [Multi-Modal Sentiment Analysis with Dynamic Attention Fusion](https://mlsanity.com/paper/2509.22729/) (2025)

## Abstract (arXiv)

Sentiment analysis, mostly based on text, has been rapidly developing in the last decade and has attracted widespread attention in both academia and industry. However, the information in the real world usually comes from multiple modalities, such as audio and text. Therefore, in this paper, based on audio and text, we consider the task of multimodal sentiment analysis and propose a novel fusion strategy including both multi-feature fusion and multi-modality fusion to improve the accuracy of audio-text sentiment analysis. We call it the DFF-ATMF (Deep Feature Fusion - Audio and Text Modality Fusion) model, which consists of two parallel branches, the audio modality based branch and the text modality based branch. Its core mechanisms are the fusion of multiple feature vectors and multiple modality attention. Experiments on the CMU-MOSI dataset and the recently released CMU-MOSEI dataset, both collected from YouTube for sentiment analysis, show the very competitive results of our DFF-ATMF model. Furthermore, by virtue of attention weight distribution heatmaps, we also demonstrate the deep features learned by using DFF-ATMF are complementary to each other and robust. Surprisingly, DFF-ATMF also achieves new state-of-the-art results on the IEMOCAP dataset, indicating that the proposed fusion strategy also has a good generalization ability for multimodal emotion recognition.