# DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations

arXiv: https://arxiv.org/abs/2310.11374 · HTML: https://mlsanity.com/paper/2310.11374/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 61 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](https://mlsanity.com/paper/2303.14070/) (2023)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [Large Language Models Understand and Can be Enhanced by Emotional Stimuli](https://mlsanity.com/paper/2307.11760/) (2023)
- [Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks](https://mlsanity.com/paper/1708.04299/) (2017)
- [Directed Acyclic Graph Network for Conversational Emotion Recognition](https://mlsanity.com/paper/2105.12907/) (2021)
- [Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks](https://mlsanity.com/paper/2305.14201/) (2023)
- [DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition](https://mlsanity.com/paper/2012.08695/) (2020)
- [A Survey of Knowledge Enhanced Pre-trained Language Models](https://mlsanity.com/paper/2211.05994/) (2022)
- [DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations](https://mlsanity.com/paper/2106.01978/) (2021)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2211.11256/) (2022)
- [Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection](https://mlsanity.com/paper/2106.01071/) (2021)
- [Multimodal Learning using Optimal Transport for Sarcasm and Humor Detection](https://mlsanity.com/paper/2110.10949/) (2021)
- [Contrast and Generation Make BART a Good Dialogue Emotion Recognizer](https://mlsanity.com/paper/2112.11202/) (2021)
- [COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition](https://mlsanity.com/paper/2206.05833/) (2022)
- [GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition](https://mlsanity.com/paper/2207.12261/) (2022)
- [Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversations](https://mlsanity.com/paper/2306.01505/) (2023)
- [Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems](https://mlsanity.com/paper/2109.04314/) (2021)
- [Multi-Task Learning with Auxiliary Speaker Identification for Conversational Emotion Recognition](https://mlsanity.com/paper/2003.01478/) (2020)
- [Multitask Learning from Augmented Auxiliary Data for Improving Speech Emotion Recognition](https://mlsanity.com/paper/2207.05298/) (2022)
- [A Quantum Probability Driven Framework for Joint Multi-Modal Sarcasm, Sentiment and Emotion Analysis](https://mlsanity.com/paper/2306.03650/) (2023)

## Built on by (most cited first, 30 of 30)

- [A Survey on LoRA of Large Language Models](https://mlsanity.com/paper/2407.11046/) (2024)
- [EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis](https://mlsanity.com/paper/2401.08508/) (2024)
- [Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought](https://mlsanity.com/paper/2401.06836/) (2024)
- [VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning](https://mlsanity.com/paper/2404.07078/) (2024)
- [Large Vision-Language Models as Emotion Recognizers in Context Awareness](https://mlsanity.com/paper/2407.11300/) (2024)
- [Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotion](https://mlsanity.com/paper/2510.04064/) (2025)
- [Both Matter: Enhancing the Emotional Intelligence of Large Language Models without Compromising the General Intelligence](https://mlsanity.com/paper/2402.10073/) (2024)
- [Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects](https://mlsanity.com/paper/2505.20511/) (2025)
- [The Good, The Bad, and Why: Unveiling Emotions in Generative AI](https://mlsanity.com/paper/2312.11111/) (2023)
- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models](https://mlsanity.com/paper/2505.11405/) (2025)
- [Training A Small Emotional Vision Language Model for Visual Art Comprehension](https://mlsanity.com/paper/2403.11150/) (2024)
- [Emotion-Anchored Contrastive Learning Framework for Emotion Recognition in Conversation](https://mlsanity.com/paper/2403.20289/) (2024)
- [Recognizing Emotion Regulation Strategies from Human Behavior with Large Language Models](https://mlsanity.com/paper/2408.04420/) (2024)
- [Evaluating Large Language Models on Financial Report Summarization: An Empirical Study](https://mlsanity.com/paper/2411.06852/) (2024)
- [Chart-to-Experience: Benchmarking Multimodal LLMs for Predicting Experiential Impact of Charts](https://mlsanity.com/paper/2505.17374/) (2025)
- [Are MLMs Trapped in the Visual Room?](https://mlsanity.com/paper/2505.23272/) (2025)
- [When LLMs Team Up: The Emergence of Collaborative Affective Computing](https://mlsanity.com/paper/2506.01698/) (2025)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [Lotus at SemEval-2025 Task 11: RoBERTa with Llama-3 Generated Explanations for Multi-Label Emotion Classification](https://mlsanity.com/paper/2502.19935/) (2025)
- [CAPE: A Chinese Dataset for Appraisal-based Emotional Generation using Large Language Models](https://mlsanity.com/paper/2410.14145/) (2024)
- [BeMERC: Behavior-Aware MLLM-based Framework for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2503.23990/) (2025)
- [Following the TRACE: A Structured Path to Empathetic Response Generation with Multi-Agent Models](https://mlsanity.com/paper/2509.21849/) (2025)
- [Unraveling Emotions with Pre-Trained Models](https://mlsanity.com/paper/2510.19668/) (2025)
- [DialogGraph-LLM: Graph-Informed LLMs for End-to-End Audio Dialogue Intent Recognition](https://mlsanity.com/paper/2511.11000/) (2025)
- [A Customer Journey in the Land of Oz: Leveraging the Wizard of Oz Technique to Model Emotions in Customer Service Interactions](https://mlsanity.com/paper/2511.21909/) (2025)
- [Large language models for spreading dynamics in complex systems](https://mlsanity.com/paper/2602.08085/) (2026)
- [A Hormone-inspired Emotion Layer for Transformer language models (HELT)](https://mlsanity.com/paper/2605.13858/) (2026)
- [VISAFF: Speaker-Centered Visual Affective Feature Learning for Emotion Recognition in Conversation](https://mlsanity.com/paper/2605.18547/) (2026)
- [Emo-Jev: Probabilistic Reasoning for Emotion Classification with Jev](https://mlsanity.com/paper/2610.08829/) (2026)

## Abstract (arXiv)

Large language models (LLMs) and their variants have shown extraordinary efficacy across numerous downstream natural language processing (NLP) tasks, which has presented a new vision for the development of NLP. Despite their remarkable performance in natural language generating (NLG), LLMs lack a distinct focus on the emotion understanding domain. As a result, using LLMs for emotion recognition may lead to suboptimal and inadequate precision. Another limitation of LLMs is that they are typical trained without leveraging multi-modal information. To overcome these limitations, we propose DialogueLLM, a context and emotion knowledge tuned LLM that is obtained by fine-tuning LLaMA models with 13,638 multi-modal (i.e., texts and videos) emotional dialogues. The visual information is considered as the supplementary knowledge to construct high-quality instructions. We offer a comprehensive evaluation of our proposed model on three benchmarking emotion recognition in conversations (ERC) datasets and compare the results against the SOTA baselines and other SOTA LLMs. Additionally, DialogueLLM-7B can be easily trained using LoRA on a 40GB A100 GPU in 5 hours, facilitating reproducibility for other researchers.