mlsanityData to 14 Oct 2026

CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

arXiv 2207.14087 ↗First posted 28 Jul 2022cs.MM cs.CL cs.CVcited by 40

Not on the map. Why?

Abstract (arXiv)

Multimodal sentiment analysis and depression estimation are two important research topics that aim to predict human mental states using multimodal data. Previous research has focused on developing effective fusion strategies for exchanging and integrating mind-related information from different modalities. Some MLP-based techniques have recently achieved considerable success in a variety of computer vision tasks. Inspired by this, we explore multimodal approaches with a feature-mixing perspective in this study. To this end, we introduce CubeMLP, a multimodal feature processing framework based entirely on MLP. CubeMLP consists of three independent MLP units, each of which has two affine transformations. CubeMLP accepts all relevant modality features as input and mixes them across three axes. After extracting the characteristics using CubeMLP, the mixed multimodal features are flattened for task predictions. Our experiments are conducted on sentiment analysis datasets: CMU-MOSI and CMU-MOSEI, and depression estimation dataset: AVEC2019. The results show that CubeMLP can achieve state-of-the-art performance with a much lower computing cost.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 28 Jul 2022.

Builds on

25 of 43 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2016Densely Connected Convolutional Networkscited by 6,720
  8. 2021ViViT: A Video Vision Transformercited by 1,620
  9. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  10. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  11. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 433
  12. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 339
13 more
  1. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  2. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 168
  3. 2018Multi-attention Recurrent Network for Human Communication Comprehensioncited by 89
  4. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
  5. 2019Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysiscited by 82
  6. 2021Hire-MLP: Vision MLP via Hierarchical Rearrangementcited by 66
  7. 2020A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysiscited by 46
  8. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  9. 2020TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysiscited by 40
  10. 2018Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignmentcited by 20
  11. 2019Multi-level Attention network using text, audio and video for Depression Predictioncited by 19
  12. 2020Low Rank Fusion based Transformers for Multimodal Sequencescited by 14
  13. 2020Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognitioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 12; 2025: 10; 2026: 15.

40 papers.

Sort:

By area:

  1. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  2. 2024Towards Multimodal Sentiment Analysis Debiasing via Bias Purificationcited by 27
  3. 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 24
  4. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  5. 2025Boosting Multimodal Learning via Disentangled Gradient Learningcited by 16
  6. 2023Emotion fusion for mental illness detection from social media: A surveycited by 13
  7. 2023General Debiasing for Multimodal Sentiment Analysiscited by 10
  8. 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
  9. 2025Enhancing Depression Detection with Chain-of-Thought Prompting: From Emotion to Reasoning Using Large Language Modelscited by 8
  10. 2022Depression Diagnosis and Analysis via Multimodal Multi-order Factor Fusioncited by 7
  11. 2024Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysiscited by 7
  12. 2024Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalitiescited by 5
28 more
  1. 2024Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detectioncited by 4
  2. 2025Enhancing Depression Detection via Question-wise Modality Fusioncited by 3
  3. 2025MLlm-DR: Towards Explainable Depression Recognition with MultiModal Large Language Modelscited by 3
  4. 2024Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baselinecited by 2
  5. 2024Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysiscited by 2
  6. 2025Automatic Depression Assessment using Machine Learning: A Comprehensive Surveycited by 2
  7. 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 2
  8. 2024Memory-Inspired Temporal Prompt Interaction for Text-Image Classificationcited by 1
  9. 2026Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 1
  10. 2026Dynamic Summary Generation for Interpretable Multimodal Depression Detectioncited by 1
  11. 2026Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysiscited by 1
  12. 2024Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUscited by 0
  13. 2024Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classificationcited by 0
  14. 2025Reliable Multimodal Learning Via Multi-Level Adaptive DeConfusioncited by 0
  15. 2025Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 0
  16. 2025AIM: Adaptive Intra-Network Modulation for Balanced Multimodal Learningcited by 0
  17. 2025FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysiscited by 0
  18. 2026GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysiscited by 0
  19. 2026Depression Detection Based on Electroencephalography Using a Hybrid Deep Neural Network CNN-GRU and MRMR Feature Selectioncited by 0
  20. 2026MA-DLE: Speech-based Automatic Depression Level Estimation via Memory Augmentationcited by 0
  21. 2026Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wildcited by 0
  22. 2026DynaBridge: Dynamic Summary-Guided Cross-Task Multimodal Fusion for DASS-Structured Mental Health Assessmentcited by 0
  23. 2026SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidencecited by 0
  24. 2026Semantic-Aligned Structural Abstraction for Multimodal Sentiment Analysiscited by 0
  25. 2026Contrastive Mixed Prompt Learning for Incomplete Multimodal Sentiment Analysis with Unseen Modality Combinationcited by 0
  26. 2026Robust Multimodal Sentiment Analysis with Incomplete Modalities via Semantic-aware Completeness based Reconstructioncited by 0
  27. 2026SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Datacited by 0
  28. 2026InstMoE: Adaptive Multimodal Routing with Specialized Expertscited by 0