mlsanityData to 7 Oct 2026

Segregate, Refine, Integrate: Decomposing Multimodal Fusion for Sentiment Analysis

arXiv 2607.12686 ↗First posted 14 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal fusion must simultaneously refine modality-specific signals and model cross-modal interactions; two competing objectives typically entangled within the same operation. We propose SeRIn (Segregate, Refine, Integrate), a multimodal LM fusion scheme that enforces this separation as an architectural prior. Modality-specific representations evolve along isolated pathways, each refined against its respective encoder context, while a dedicated cross-modal pathway accumulates their joint evolution without contaminating unimodal streams. Full cross-modal interaction is deferred to a final prediction step - ablations confirm that structured interactions, not added capacity, drive the gains; gate analysis under visual corruption reveals emergent modality reweighting without explicit supervision. SeRIn achieves state-of-the-art results on CH-SIMS and CMU-MOSEI, improving all metrics on both benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 39 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

19 of 39 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  4. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  5. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  6. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  7. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
  8. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  9. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 262
  10. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  11. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  12. 2019UER: An Open-Source Toolkit for Pre-training Modelscited by 61
7 more
  1. 2019An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Modelscited by 39
  2. 2024DLF: Disentangled-Language-Focused Multimodal Sentiment Analysiscited by 27
  3. 2024Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Contentcited by 25
  4. 2022M-SENA: An Integrated Platform for Multimodal Sentiment Analysiscited by 21
  5. 2024Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysiscited by 7
  6. 2023SeqAug: Sequential Feature Resampling as a modality agnostic augmentation methodcited by 3
  7. 2025DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysiscited by 3

Built on by

Not cited yet.