# Transfer-based adaptive tree for multimodal sentiment analysis based on user latent aspects

arXiv: https://arxiv.org/abs/2106.14174 · HTML: https://mlsanity.com/paper/2106.14174/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 100 references that are in the map.
- First posted on arXiv 27 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2102.04830/) (2021)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Reasoning about Entailment with Neural Attention](https://mlsanity.com/paper/1509.06664/) (2015)
- [Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications](https://mlsanity.com/paper/1606.03237/) (2016)
- [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](https://mlsanity.com/paper/1806.06228/) (2018)
- [Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis](https://mlsanity.com/paper/1905.05812/) (2019)
- [Learning Content-Weighted Deep Image Compression](https://mlsanity.com/paper/1904.00664/) (2019)
- [Progressive Neural Networks for Transfer Learning in Emotion Recognition](https://mlsanity.com/paper/1706.03256/) (2017)
- [On the Challenges of Sentiment Analysis for Dynamic Events](https://mlsanity.com/paper/1710.02514/) (2017)
- [SoulMate: Short-text author linking through Multi-aspect temporal-textual embedding](https://mlsanity.com/paper/1910.12180/) (2019)
- [BrePartition: Optimized High-Dimensional kNN Search with Bregman Distances](https://mlsanity.com/paper/2006.00227/) (2020)
- [TEAGS: Time-aware Text Embedding Approach to Generate Subgraphs](https://mlsanity.com/paper/1907.03191/) (2019)
- [Emotion-Based Crowd Simulation Model Based on Physical Strength Consumption for Emergency Scenarios](https://mlsanity.com/paper/1801.00216/) (2017)

## Built on by (most cited first, 4 of 4)

- [Recent Advances in Multimodal Affective Computing: An NLP Perspective](https://mlsanity.com/paper/2409.07388/) (2024)
- [Multiverse: Multilingual Evidence for Fake News Detection](https://mlsanity.com/paper/2211.14279/) (2022)
- [Multi-Modality Collaborative Learning for Sentiment Analysis](https://mlsanity.com/paper/2501.12424/) (2025)
- [Modeling Induced Pleasure through Cognitive Appraisal Prediction via Multimodal Fusion](https://mlsanity.com/paper/2604.23753/) (2026)

## Abstract (arXiv)

Multimodal sentiment analysis benefits various applications such as human-computer interaction and recommendation systems. It aims to infer the users' bipolar ideas using visual, textual, and acoustic signals. Although researchers affirm the association between cognitive cues and emotional manifestations, most of the current multimodal approaches in sentiment analysis disregard user-specific aspects. To tackle this issue, we devise a novel method to perform multimodal sentiment prediction using cognitive cues, such as personality. Our framework constructs an adaptive tree by hierarchically dividing users and trains the LSTM-based submodels, utilizing an attention-based fusion to transfer cognitive-oriented knowledge within the tree. Subsequently, the framework consumes the conclusive agglomerative knowledge from the adaptive tree to predict final sentiments. We also devise a dynamic dropout method to facilitate data sharing between neighboring nodes, reducing data sparsity. The empirical results on real-world datasets determine that our proposed model for sentiment prediction can surpass trending rivals. Moreover, compared to other ensemble approaches, the proposed transfer-based algorithm can better utilize the latent cognitive cues and foster the prediction outcomes. Based on the given extrinsic and intrinsic analysis results, we note that compared to other theoretical-based techniques, the proposed hierarchical clustering approach can better group the users within the adaptive tree.