mlsanityData to 7 Oct 2026

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

arXiv 1806.00064 ↗First posted 31 May 2018cs.AI cs.LG stat.MLcited by 268

Not on the map. Why?

Abstract (arXiv)

Multimodal research is an emerging field of artificial intelligence, and one of the main research problems in this field is multimodal fusion. The fusion of multimodal data is the process of integrating multiple unimodal representations into one compact multimodal representation. Previous research in this field has exploited the expressiveness of tensors for multimodal representation. However, these methods often suffer from exponential increase in dimensions and in computational complexity introduced by transformation of input into tensor. In this paper, we propose the Low-rank Multimodal Fusion method, which performs multimodal fusion using low-rank tensors to improve efficiency. We evaluate our model on three different tasks: multimodal sentiment analysis, speaker trait analysis, and emotion recognition. Our model achieves competitive results on all these tasks while drastically reducing computational complexity. Additional experiments also show that our model can perform robustly for a wide range of low-rank settings, and is indeed much more efficient in both training and inference compared to other methods that utilize tensor representations.

Key facts. Cited by 268 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 40; 2023: 45; 2024: 51; 2025: 54; 2026: 49.

268 papers.

Sort:

By area:

  1. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 243
  2. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 183
  3. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  4. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  5. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  6. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  7. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 81
  8. 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
  9. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  10. 2024Multimodal Alignment and Fusion: A Surveycited by 73
  11. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 67
  12. 2022UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognitioncited by 61
38 more
  1. 2022Dynamic Multimodal Fusioncited by 54
  2. 2022MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversationscited by 52
  3. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  4. 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
  5. 2023Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversationscited by 47
  6. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  7. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  8. 2022Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Modulecited by 43
  9. 2021Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysiscited by 41
  10. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  11. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  12. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  13. 2022Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modelingcited by 33
  14. 2024Revisiting Multi-modal Emotion Learning with Broad State Space Models and Probability-guidance Fusioncited by 33
  15. 2024Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrumcited by 33
  16. 2023A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learningcited by 32
  17. 2024Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Modelcited by 30
  18. 2021M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysiscited by 29
  19. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29
  20. 2023A Low-rank Matching Attention based Cross-modal Feature Fusion Method for Conversational Emotion Recognitioncited by 29
  21. 2025I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Expertscited by 29
  22. 2023Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptationcited by 27
  23. 2023Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysiscited by 27
  24. 2024DLF: Disentangled-Language-Focused Multimodal Sentiment Analysiscited by 27
  25. 2021How to Leverage Multimodal EHR Data for Better Medical Predictions?cited by 26
  26. 2023MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognitioncited by 26
  27. 2023Multimodal Multi-loss Fusion Network for Sentiment Analysiscited by 26
  28. 2024Towards Multimodal Sentiment Analysis Debiasing via Bias Purificationcited by 26
  29. 2024MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversationscited by 25
  30. 2022i-Code: An Integrative and Composable Multimodal Learning Frameworkcited by 24
  31. 2021Expansion-Squeeze-Excitation Fusion Network for Elderly Activity Recognitioncited by 23
  32. 2023DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognitioncited by 23
  33. 2024Efficient Long-distance Latent Relation-aware Graph Neural Network for Multi-modal Emotion Recognition in Conversationscited by 23
  34. 2022Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognitioncited by 22
  35. 2022COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognitioncited by 22
  36. 2022M-SENA: An Integrated Platform for Multimodal Sentiment Analysiscited by 21
  37. 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
  38. 2022Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generationcited by 20

50 most cited of 268 All 268 (JSON)