# CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

arXiv: https://arxiv.org/abs/2207.14087 · HTML: https://mlsanity.com/paper/2207.14087/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 28 Jul 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [Memory Fusion Network for Multi-view Sequential Learning](https://mlsanity.com/paper/1802.00927/) (2018)
- [Multi-attention Recurrent Network for Human Communication Comprehension](https://mlsanity.com/paper/1802.00923/) (2018)
- [AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition](https://mlsanity.com/paper/1907.11510/) (2019)
- [Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis](https://mlsanity.com/paper/1911.05544/) (2019)
- [Hire-MLP: Vision MLP via Hierarchical Rearrangement](https://mlsanity.com/paper/2108.13341/) (2021)
- [A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis](https://mlsanity.com/paper/2006.15955/) (2020)
- [Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis](https://mlsanity.com/paper/2107.13669/) (2021)
- [TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2009.02902/) (2020)
- [Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment](https://mlsanity.com/paper/1805.08660/) (2018)
- [Multi-level Attention network using text, audio and video for Depression Prediction](https://mlsanity.com/paper/1909.01417/) (2019)
- [Low Rank Fusion based Transformers for Multimodal Sequences](https://mlsanity.com/paper/2007.02038/) (2020)
- [Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition](https://mlsanity.com/paper/2010.02057/) (2020)

## Built on by (most cited first, 30 of 40)

- [Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2404.16456/) (2024)
- [Towards Multimodal Sentiment Analysis Debiasing via Bias Purification](https://mlsanity.com/paper/2403.05023/) (2024)
- [DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning](https://mlsanity.com/paper/2503.11892/) (2025)
- [Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning](https://mlsanity.com/paper/2411.02793/) (2024)
- [Boosting Multimodal Learning via Disentangled Gradient Learning](https://mlsanity.com/paper/2507.10213/) (2025)
- [Emotion fusion for mental illness detection from social media: A survey](https://mlsanity.com/paper/2304.09493/) (2023)
- [General Debiasing for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2307.10511/) (2023)
- [Debiased Multimodal Understanding for Human Language Sequences](https://mlsanity.com/paper/2403.05025/) (2024)
- [Enhancing Depression Detection with Chain-of-Thought Prompting: From Emotion to Reasoning Using Large Language Models](https://mlsanity.com/paper/2502.05879/) (2025)
- [Depression Diagnosis and Analysis via Multimodal Multi-order Factor Fusion](https://mlsanity.com/paper/2301.00254/) (2022)
- [Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2410.04491/) (2024)
- [Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities](https://mlsanity.com/paper/2407.06125/) (2024)
- [Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection](https://mlsanity.com/paper/2402.09055/) (2024)
- [Enhancing Depression Detection via Question-wise Modality Fusion](https://mlsanity.com/paper/2503.20496/) (2025)
- [MLlm-DR: Towards Explainable Depression Recognition with MultiModal Large Language Models](https://mlsanity.com/paper/2507.05591/) (2025)
- [Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baseline](https://mlsanity.com/paper/2407.06115/) (2024)
- [Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2408.16029/) (2024)
- [Automatic Depression Assessment using Machine Learning: A Comprehensive Survey](https://mlsanity.com/paper/2506.18915/) (2025)
- [CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusion](https://mlsanity.com/paper/2602.19140/) (2026)
- [Memory-Inspired Temporal Prompt Interaction for Text-Image Classification](https://mlsanity.com/paper/2401.14856/) (2024)
- [Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2603.09111/) (2026)
- [Dynamic Summary Generation for Interpretable Multimodal Depression Detection](https://mlsanity.com/paper/2604.11334/) (2026)
- [Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2604.12518/) (2026)
- [Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs](https://mlsanity.com/paper/2403.17607/) (2024)
- [Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classification](https://mlsanity.com/paper/2412.14489/) (2024)
- [Reliable Multimodal Learning Via Multi-Level Adaptive DeConfusion](https://mlsanity.com/paper/2502.19674/) (2025)
- [Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2506.01490/) (2025)
- [AIM: Adaptive Intra-Network Modulation for Balanced Multimodal Learning](https://mlsanity.com/paper/2508.19769/) (2025)
- [FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis](https://mlsanity.com/paper/2510.16086/) (2025)
- [GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2601.09606/) (2026)

## Abstract (arXiv)

Multimodal sentiment analysis and depression estimation are two important research topics that aim to predict human mental states using multimodal data. Previous research has focused on developing effective fusion strategies for exchanging and integrating mind-related information from different modalities. Some MLP-based techniques have recently achieved considerable success in a variety of computer vision tasks. Inspired by this, we explore multimodal approaches with a feature-mixing perspective in this study. To this end, we introduce CubeMLP, a multimodal feature processing framework based entirely on MLP. CubeMLP consists of three independent MLP units, each of which has two affine transformations. CubeMLP accepts all relevant modality features as input and mixes them across three axes. After extracting the characteristics using CubeMLP, the mixed multimodal features are flattened for task predictions. Our experiments are conducted on sentiment analysis datasets: CMU-MOSI and CMU-MOSEI, and depression estimation dataset: AVEC2019. The results show that CubeMLP can achieve state-of-the-art performance with a much lower computing cost.