# DialogueTRM: Exploring the Intra- and Inter-Modal Emotional Behaviors in the Conversation

arXiv: https://arxiv.org/abs/2010.07637 · HTML: https://mlsanity.com/paper/2010.07637/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 13 of 13)

- [EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition](https://mlsanity.com/paper/2203.13504/) (2022)
- [M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2206.02187/) (2022)
- [UniSA: Unified Generative Framework for Sentiment Analysis](https://mlsanity.com/paper/2309.01339/) (2023)
- [Deep Emotion Recognition in Textual Conversations: A Survey](https://mlsanity.com/paper/2211.09172/) (2022)
- [Emotion Dynamics Modeling via BERT](https://mlsanity.com/paper/2104.07252/) (2021)
- [Contextual Information and Commonsense Based Prompt for Emotion Recognition in Conversation](https://mlsanity.com/paper/2207.13254/) (2022)
- [SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model](https://mlsanity.com/paper/2106.08785/) (2021)
- [Grounding Emotion Recognition with Visual Prototypes: VEGA -- Revisiting CLIP in MERC](https://mlsanity.com/paper/2508.06564/) (2025)
- [WavFusion: Towards wav2vec 2.0 Multimodal Speech Emotion Recognition](https://mlsanity.com/paper/2412.05558/) (2024)
- [TED: Turn Emphasis with Dialogue Feature Attention for Emotion Recognition in Conversation](https://mlsanity.com/paper/2501.01123/) (2025)
- [ERFC: Happy Customers with Emotion Recognition and Forecasting in Conversation in Call Centers](https://mlsanity.com/paper/2509.18175/) (2025)
- [Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion](https://mlsanity.com/paper/2511.13731/) (2025)
- [Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversation emotion recognition](https://mlsanity.com/paper/2603.25752/) (2026)

## Abstract (arXiv)

Emotion Recognition in Conversations (ERC) is essential for building empathetic human-machine systems. Existing studies on ERC primarily focus on summarizing the context information in a conversation, however, ignoring the differentiated emotional behaviors within and across different modalities. Designing appropriate strategies that fit the differentiated multi-modal emotional behaviors can produce more accurate emotional predictions. Thus, we propose the DialogueTransformer to explore the differentiated emotional behaviors from the intra- and inter-modal perspectives. For intra-modal, we construct a novel Hierarchical Transformer that can easily switch between sequential and feed-forward structures according to the differentiated context preference within each modality. For inter-modal, we constitute a novel Multi-Grained Interactive Fusion that applies both neuron- and vector-grained feature interactions to learn the differentiated contributions across all modalities. Experimental results show that DialogueTRM outperforms the state-of-the-art by a significant margin on three benchmark datasets.