# Affect Recognition in Conversations Using Large Language Models

arXiv: https://arxiv.org/abs/2309.12881 · HTML: https://mlsanity.com/paper/2309.12881/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge](https://mlsanity.com/paper/1605.01600/) (2016)
- [Emotional Intelligence of Large Language Models](https://mlsanity.com/paper/2307.09042/) (2023)
- [Is ChatGPT Equipped with Emotional Dialogue Capabilities?](https://mlsanity.com/paper/2304.09582/) (2023)
- [Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench](https://mlsanity.com/paper/2308.03656/) (2023)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [Topic Modeling Based Multi-modal Depression Detection](https://mlsanity.com/paper/1803.10384/) (2018)
- [Emotion recognition by fusing time synchronous and time asynchronous representations](https://mlsanity.com/paper/2010.14102/) (2020)
- [EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2109.04919/) (2021)
- [Self-supervised representations in speech-based depression detection](https://mlsanity.com/paper/2305.12263/) (2023)
- [A Step Towards Preserving Speakers' Identity While Detecting Depression Via Speaker Disentanglement](https://mlsanity.com/paper/2206.09530/) (2022)
- [From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue](https://mlsanity.com/paper/2308.12648/) (2023)

## Built on by (most cited first, 9 of 9)

- [Emotion Detection for Misinformation: A Review](https://mlsanity.com/paper/2311.00671/) (2023)
- [From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs](https://mlsanity.com/paper/2408.17026/) (2024)
- [AMPLE: Emotion-Aware Multimodal Fusion Prompt Learning for Fake News Detection](https://mlsanity.com/paper/2410.15591/) (2024)
- [When LLMs Team Up: The Emergence of Collaborative Affective Computing](https://mlsanity.com/paper/2506.01698/) (2025)
- [Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?](https://mlsanity.com/paper/2508.05474/) (2025)
- [SocialLM: Social Signal Processing of Patient-Provider Communication using LLMs and Contextual Aggregation](https://mlsanity.com/paper/2505.04152/) (2025)
- [Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration](https://mlsanity.com/paper/2606.18259/) (2026)
- [Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models](https://mlsanity.com/paper/2602.00123/) (2026)
- [NTDH: Complex Reasoning for Comprehensive Affective Analysis](https://mlsanity.com/paper/2608.06425/) (2026)

## Abstract (arXiv)

Affect recognition, encompassing emotions, moods, and feelings, plays a pivotal role in human communication. In the realm of conversational artificial intelligence, the ability to discern and respond to human affective cues is a critical factor for creating engaging and empathetic interactions. This study investigates the capacity of large language models (LLMs) to recognise human affect in conversations, with a focus on both open-domain chit-chat dialogues and task-oriented dialogues. Leveraging three diverse datasets, namely IEMOCAP (Busso et al., 2008), EmoWOZ (Feng et al., 2022), and DAIC-WOZ (Gratch et al., 2014), covering a spectrum of dialogues from casual conversations to clinical interviews, we evaluate and compare LLMs' performance in affect recognition. Our investigation explores the zero-shot and few-shot capabilities of LLMs through in-context learning as well as their model capacities through task-specific fine-tuning. Additionally, this study takes into account the potential impact of automatic speech recognition errors on LLM predictions. With this work, we aim to shed light on the extent to which LLMs can replicate human-like affect recognition capabilities in conversations.