mlsanityData to 7 Oct 2026

Fine-Grained Visual Preprocessing and Dual-Stream Temporal Modeling for Multimodal Sentiment Analysis on Social Media

arXiv 2609.07010 ↗First posted 7 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal sentiment analysis often remains text-dominant due to raw-video noise and insufficient temporal modeling. Using CH-SIMS v2.0S, this study proposes three improvements: the NAPS pipeline---a seven-stage system integrating face tracking,identity embedding, and normalized lip-motion analysis to reduce visual noise;DS-TANet, combining an EfficientNetB2 static stream, RAFT optical-flow motion stream, motion-guided attention, and Bi-GRU temporal modeling; and DS-TAFNet, fusing visual and MacBERT-Base textual representations via concatenation fusion. With NAPS, the static visual baseline achieves 80.98% Macro F1, comparable to the text baseline of 80.55%; DS-TANet improves visual Macro F1 to 82.58%;and DS-TAFNet achieves 87.49% accuracy and 87.48% Macro F1. These results demonstrate that improving visual input quality and temporal representation is more effective than increasing fusion complexity under limited-data conditions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

20 of 37 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  7. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  8. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  9. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  10. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  11. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
  12. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
8 more
  1. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  2. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  3. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 167
  4. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  5. 2022Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Modulecited by 43
  6. 2024Visual Multi-Object Tracking with Re-Identification and Occlusion Handling using Labeled Random Finite Setscited by 11
  7. 2024Precise Facial Landmark Detection by Dynamic Semantic Aggregation Transformercited by 6
  8. 2025Calibrating Multimodal Consensus for Emotion Recognitioncited by 1

Built on by

Not cited yet.