InstructERC: Reforming Emotion Recognition in Conversation with Multi-task Retrieval-Augmented Large Language Models
On the map: Retrieval-augmented generation → Conversational and speech RAG
Abstract (arXiv)
The field of emotion recognition of conversation (ERC) has been focusing on separating sentence feature encoding and context modeling, lacking exploration in generative paradigms based on unified designs. In this study, we propose a novel approach, InstructERC, to reformulate the ERC task from a discriminative framework to a generative framework based on Large Language Models (LLMs). InstructERC makes three significant contributions: (1) it introduces a simple yet effective retrieval template module, which helps the model explicitly integrate multi-granularity dialogue supervision information. (2) We introduce two additional emotion alignment tasks, namely speaker identification and emotion prediction tasks, to implicitly model the dialogue role relationships and future emotional tendencies in conversations. (3) Pioneeringly, we unify emotion labels across benchmarks through the feeling wheel to fit real application scenarios. InstructERC still perform impressively on this unified dataset. Our LLM-based plugin framework significantly outperforms all previous models and achieves comprehensive SOTA on three commonly used ERC datasets. Extensive analysis of parameter-efficient and data-scaling experiments provides empirical guidance for applying it in practical scenarios.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 46 references that are in the map. First posted on arXiv 21 Sep 2023.
Builds on
23 of 46 references.
References · 23
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
- 2022Unified Structure Generation for Universal Information Extractioncited by 221
- 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 218
- 2019DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversationcited by 186
11 more
- 2017Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networkscited by 102
- 2021Directed Acyclic Graph Network for Conversational Emotion Recognitioncited by 97
- 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 70
- 2022M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversationcited by 47
- 2022Supervised Prototypical Contrastive Learning for Emotion Recognition in Conversationcited by 31
- 2022GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognitioncited by 20
- 2023Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversationscited by 19
- 2020Multi-Task Learning with Auxiliary Speaker Identification for Conversational Emotion Recognitioncited by 12
- 2023EmotionIC: emotional inertia and contagion-driven dependency modeling for emotion recognition in conversationcited by 7
- 2023Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attentioncited by 6
- 2023Mimicking the Thinking Process for Emotion Recognition in Conversation with Prompts and Paraphrasingcited by 4
Built on by
Citing papers per year: 2024: 1; 2025: 7; 2026: 3.
11 papers.
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
- 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoningcited by 32
- 2025Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Surveycited by 19
- 2025OmniVox: Zero-Shot Emotion Recognition with Omni-LLMscited by 7
- 2025Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Modelscited by 4
- 2025When LLMs Team Up: The Emergence of Collaborative Affective Computingcited by 2
- 2026SCoPE: Shift-Aware Speaker-Conditioned Priors for Emotion Recognition in Conversationscited by 2
- 2024Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Modelscited by 1
- 2025MMD-Thinker: Adaptive Multi-Dimensional Thinking for Multimodal Misinformation Detectioncited by 0
- 2026Mechanistic Decoding of Cognitive Constructs in Large Language Modelscited by 0
- 2026NTDH: Complex Reasoning for Comprehensive Affective Analysiscited by 0
- 2026NTDH: Complex Reasoning for Comprehensive Affective Analysis5 Aug 2026
- 2026SCoPE: Shift-Aware Speaker-Conditioned Priors for Emotion Recognition in Conversations13 May 2026
- 2026Mechanistic Decoding of Cognitive Constructs in Large Language Models16 Apr 2026
- 2025MMD-Thinker: Adaptive Multi-Dimensional Thinking for Multimodal Misinformation Detection17 Nov 2025
- 2025Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey29 Sep 2025
- 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning14 Aug 2025
- 2025When LLMs Team Up: The Emergence of Collaborative Affective Computing2 Jun 2025
- 2025Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models10 Apr 2025
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems31 Mar 2025
- 2025OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs27 Mar 2025
- 2024Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Models17 Dec 2024
Similar, not cited
Close in wording, no citation either way
- 2026A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations
- 2022Emotion Recognition in Conversation using Probabilistic Soft Logic
- 2026When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI
- 2023A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations
- 2026Acoustic and Semantic Modeling of Emotion in Spoken Language ↗
- 2025Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs
- 2026EmoLASP: Emotion Recognition with Language Models and Answer Set Programming
- 2026VISAFF: Speaker-Centered Visual Affective Feature Learning for Emotion Recognition in Conversation