mlsanityData to 14 Oct 2026

Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment

arXiv 1805.08660 ↗First posted 22 May 2018cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Multimodal affective computing, learning to recognize and interpret human affects and subjective information from multiple data sources, is still challenging because: (i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract level, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utter-ance-level sentiment and emotion from text and audio data. Our introduced model outperforms the state-of-the-art approaches on published datasets and we demonstrated that our model is able to visualize and interpret the synchronized attention over modalities.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 5; 2023: 4; 2024: 4; 2025: 3.

20 papers.

Sort:

By area:

  1. 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 119
  2. 2022Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentationcited by 47
  3. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  4. 2025AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Modelscited by 44
  5. 2022CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimationcited by 40
  6. 2024InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulationcited by 22
  7. 2022Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysiscited by 20
  8. 2024Masked Graph Learning with Recurrent Alignment for Multimodal Emotion Recognition in Conversationcited by 19
  9. 2023SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processingcited by 16
  10. 2021ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptorscited by 12
  11. 2024Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Spacecited by 8
  12. 2025CogniAlign: Word-Level Multimodal Speech Alignment with Gated Cross-Attention for Alzheimer's Detectioncited by 7
8 more
  1. 2021Graph Capsule Aggregation for Unaligned Multimodal Sequencescited by 6
  2. 2022MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysiscited by 6
  3. 2023Towards Interpretability in Audio and Visual Affective Machine Learning: A Reviewcited by 3
  4. 2025DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysiscited by 3
  5. 2022Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysiscited by 2
  6. 2023Complex Mathematical Symbol Definition Structures: A Dataset and Model for Coordination Resolution in Definition Extractioncited by 1
  7. 2023PowMix: A Versatile Regularizer for Multimodal Sentiment Analysiscited by 1
  8. 2021Dialog speech sentiment classification for imbalanced datasetscited by 0