mlsanityData to 7 Oct 2026

Tensor Fusion Network for Multimodal Sentiment Analysis

arXiv 1707.07250 ↗First posted 23 Jul 2017cs.CLcited by 429

Not on the map. Why?

Abstract (arXiv)

Multimodal sentiment analysis is an increasingly popular research area, which extends the conventional language-based definition of sentiment analysis to a multimodal setup where other relevant modalities accompany language. In this paper, we pose the problem of multimodal sentiment analysis as modeling intra-modality and inter-modality dynamics. We introduce a novel model, termed Tensor Fusion Network, which learns both such dynamics end-to-end. The proposed approach is tailored for the volatile nature of spoken language in online videos as well as accompanying gestures and voice. In the experiments, our model outperforms state-of-the-art approaches for both multimodal and unimodal sentiment analysis.

Key facts. Cited by 429 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 55; 2022: 64; 2023: 63; 2024: 77; 2025: 79; 2026: 91.

429 papers.

Sort:

By area:

  1. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 243
  2. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 183
  3. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  4. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  5. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  6. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  7. 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
  8. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  9. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 81
  10. 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
  11. 2024Multimodal Alignment and Fusion: A Surveycited by 73
  12. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Expertscited by 69
38 more
  1. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 67
  2. 2023Align and Attend: Multimodal Summarization with Dual Contrastive Lossescited by 65
  3. 2022UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognitioncited by 61
  4. 2022Dynamic Multimodal Fusioncited by 54
  5. 2022MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversationscited by 52
  6. 2022Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Groundingcited by 52
  7. 2022EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognitioncited by 48
  8. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  9. 2023Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversationscited by 47
  10. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  11. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  12. 2021MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognitioncited by 44
  13. 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
  14. 2022Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Modulecited by 43
  15. 2025AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Modelscited by 43
  16. 2021Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysiscited by 41
  17. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  18. 2021Multimodal End-to-End Sparse Model for Emotion Recognitioncited by 40
  19. 2022CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimationcited by 39
  20. 2023Transformers in Speech Processing: A Surveycited by 38
  21. 2023GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognitioncited by 38
  22. 2022Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Reviewcited by 37
  23. 2023A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversationscited by 37
  24. 2021Multimodal Learning using Optimal Transport for Sarcasm and Humor Detectioncited by 36
  25. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  26. 2021CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversationcited by 35
  27. 2022Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modelingcited by 33
  28. 2024Revisiting Multi-modal Emotion Learning with Broad State Space Models and Probability-guidance Fusioncited by 33
  29. 2024Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrumcited by 33
  30. 2021Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarizationcited by 32
  31. 2023A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learningcited by 32
  32. 2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
  33. 2024HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognitioncited by 30
  34. 2024Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Modelcited by 30
  35. 2021Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Datacited by 29
  36. 2021M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysiscited by 29
  37. 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
  38. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29

50 most cited of 429 All 429 (JSON)