# TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation

arXiv: https://arxiv.org/abs/2401.12987 · HTML: https://mlsanity.com/paper/2401.12987/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 29 of 29)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [A Comprehensive Overhaul of Feature Distillation](https://mlsanity.com/paper/1904.01866/) (2019)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [Knowledge Distillation from A Stronger Teacher](https://mlsanity.com/paper/2205.10536/) (2022)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [Cross Modal Distillation for Supervision Transfer](https://mlsanity.com/paper/1507.00448/) (2015)
- [COSMIC: COmmonSense knowledge for eMotion Identification in Conversations](https://mlsanity.com/paper/2010.02795/) (2020)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2107.06779/) (2021)
- [Directed Acyclic Graph Network for Conversational Emotion Recognition](https://mlsanity.com/paper/2105.12907/) (2021)
- [Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations](https://mlsanity.com/paper/1909.10681/) (2019)
- [DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations](https://mlsanity.com/paper/2106.01978/) (2021)
- [EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTa](https://mlsanity.com/paper/2108.12009/) (2021)
- [MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations](https://mlsanity.com/paper/2203.02385/) (2022)
- [EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition](https://mlsanity.com/paper/2203.13504/) (2022)
- [M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2206.02187/) (2022)
- [The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation](https://mlsanity.com/paper/2206.06487/) (2022)
- [HiGRU: Hierarchical Gated Recurrent Units for Utterance-level Emotion Recognition](https://mlsanity.com/paper/1904.04446/) (2019)
- [A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2310.20494/) (2023)
- [CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversation](https://mlsanity.com/paper/2108.11626/) (2021)
- [Supervised Prototypical Contrastive Learning for Emotion Recognition in Conversation](https://mlsanity.com/paper/2210.08713/) (2022)
- [GA2MIF: Graph and Attention Based Two-Stage Multi-Source Information Fusion for Conversational Emotion Detection](https://mlsanity.com/paper/2207.11900/) (2022)
- [Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversations](https://mlsanity.com/paper/2306.01505/) (2023)
- [MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding](https://mlsanity.com/paper/2101.01881/) (2021)

## Built on by (most cited first, 12 of 12)

- [Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild](https://mlsanity.com/paper/2407.12927/) (2024)
- [OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs](https://mlsanity.com/paper/2503.21480/) (2025)
- [Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts](https://mlsanity.com/paper/2503.06805/) (2025)
- [Qieemo: Speech Is All You Need in the Emotion Recognition in Conversations](https://mlsanity.com/paper/2503.22687/) (2025)
- [Emotion Recognition in Signers](https://mlsanity.com/paper/2512.15376/) (2025)
- [A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2602.23300/) (2026)
- [Bimodal Connection Attention Fusion for Speech Emotion Recognition](https://mlsanity.com/paper/2503.05858/) (2025)
- [Disentangled Dual-Branch Graph Learning for Conversational Emotion Recognition](https://mlsanity.com/paper/2604.14204/) (2026)
- [To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition](https://mlsanity.com/paper/2605.04877/) (2026)
- [Emotion Recognition in Sign Language Conversation](https://mlsanity.com/paper/2605.23328/) (2026)
- [Rationale-Guided Learning for Multimodal Emotion Recognition](https://mlsanity.com/paper/2608.10448/) (2026)
- [ReH-FUSE: Reliability-Aware Hierarchical Fusion of Experts for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2609.13857/) (2026)

## Abstract (arXiv)

Emotion Recognition in Conversation (ERC) plays a crucial role in enabling dialogue systems to effectively respond to user requests. The emotions in a conversation can be identified by the representations from various modalities, such as audio, visual, and text. However, due to the weak contribution of non-verbal modalities to recognize emotions, multimodal ERC has always been considered a challenging task. In this paper, we propose Teacher-leading Multimodal fusion network for ERC (TelME). TelME incorporates cross-modal knowledge distillation to transfer information from a language model acting as the teacher to the non-verbal students, thereby optimizing the efficacy of the weak modalities. We then combine multimodal features using a shifting fusion approach in which student networks support the teacher. TelME achieves state-of-the-art performance in MELD, a multi-speaker conversation dataset for ERC. Finally, we demonstrate the effectiveness of our components through additional experiments.