mlsanityData to 14 Oct 2026

Centering Emotion Hotspots: Multimodal Local-Global Fusion and Cross-Modal Alignment for Emotion Recognition in Conversations

arXiv 2510.08606 ↗First posted 7 Oct 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Emotion Recognition in Conversations (ERC) is hard because discriminative evidence is sparse, localized, and often asynchronous across modalities. We center ERC on emotion hotspots and present a unified model that detects per-utterance hotspots in text, audio, and video, fuses them with global features via Hotspot-Gated Fusion, and aligns modalities using a routed Mixture-of-Aligners; a cross-modal graph encodes conversational structure. This design focuses modeling on salient spans, mitigates misalignment, and preserves context. Experiments on standard ERC benchmarks show consistent gains over strong baselines, with ablations confirming the contributions of HGF and MoA. Our results point to a hotspot-centric view that can inform future multimodal learning, offering a new perspective on modality fusion in ERC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

14 of 24 references.

References · 14

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  3. 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 218
  4. 2019DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversationcited by 186
  5. 2021DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversationscited by 71
  6. 2022A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advancescited by 59
  7. 2020A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysiscited by 46
  8. 2018Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modelingcited by 39
  9. 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
  10. 2020Multilogue-Net: A Context Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversationcited by 27
  11. 2023EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Modelscited by 22
  12. 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
2 more
  1. 2023Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interactioncited by 18
  2. 2022Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognitioncited by 16

Built on by

Not cited yet.