# EmoBench: Evaluating the Emotional Intelligence of Large Language Models

arXiv: https://arxiv.org/abs/2402.12071 · HTML: https://mlsanity.com/paper/2402.12071/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 68 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions](https://mlsanity.com/paper/2310.15421/) (2023)
- [Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification](https://mlsanity.com/paper/2308.07921/) (2023)
- [Large Language Models Understand and Can be Enhanced by Emotional Stimuli](https://mlsanity.com/paper/2307.11760/) (2023)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [Recognizing Emotion Cause in Conversations](https://mlsanity.com/paper/2012.11820/) (2020)
- [CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues](https://mlsanity.com/paper/2203.13926/) (2022)
- [A Wide Evaluation of ChatGPT on Affective Computing Tasks](https://mlsanity.com/paper/2308.13911/) (2023)
- [ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind](https://mlsanity.com/paper/2305.15068/) (2023)
- [Chatbots for Mental Health Support: Exploring the Impact of Emohaa on Reducing Mental Distress in China](https://mlsanity.com/paper/2209.10183/) (2022)
- [Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition](https://mlsanity.com/paper/2310.14614/) (2023)

## Built on by (most cited first, 30 of 53)

- [EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models](https://mlsanity.com/paper/2502.04424/) (2025)
- [Human-like object concept representations emerge naturally in multimodal large language models](https://mlsanity.com/paper/2407.01067/) (2024)
- [Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective](https://mlsanity.com/paper/2408.04638/) (2024)
- [EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety](https://mlsanity.com/paper/2504.09689/) (2025)
- [Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models](https://mlsanity.com/paper/2505.02847/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models](https://mlsanity.com/paper/2508.09210/) (2025)
- [SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactions](https://mlsanity.com/paper/2506.23046/) (2025)
- [From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs](https://mlsanity.com/paper/2408.17026/) (2024)
- [A Survey on Human-Centric LLMs](https://mlsanity.com/paper/2411.14491/) (2024)
- [HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue](https://mlsanity.com/paper/2601.19922/) (2026)
- [Academically intelligent LLMs are not necessarily socially intelligent](https://mlsanity.com/paper/2403.06591/) (2024)
- [MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis](https://mlsanity.com/paper/2411.11235/) (2024)
- [AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment](https://mlsanity.com/paper/2603.26680/) (2026)
- [Emergence of Hierarchical Emotion Organization in Large Language Models](https://mlsanity.com/paper/2507.10599/) (2025)
- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Models](https://mlsanity.com/paper/2511.00850/) (2025)
- [LIFELONG SOTOPIA: Evaluating Social Intelligence of Language Agents Over Lifelong Social Interactions](https://mlsanity.com/paper/2506.12666/) (2025)
- [Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoning](https://mlsanity.com/paper/2602.00971/) (2026)
- [One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence](https://mlsanity.com/paper/2602.03109/) (2026)
- [Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMs](https://mlsanity.com/paper/2603.10011/) (2026)
- [Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding](https://mlsanity.com/paper/2601.16449/) (2026)
- [Social-R1: Towards Human-like Social Reasoning in LLMs](https://mlsanity.com/paper/2603.09249/) (2026)
- [On the Adaptive Psychological Persuasion of Large Language Models](https://mlsanity.com/paper/2506.06800/) (2025)
- [MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?](https://mlsanity.com/paper/2506.13065/) (2025)
- [Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs](https://mlsanity.com/paper/2508.16921/) (2025)
- [MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue](https://mlsanity.com/paper/2603.06194/) (2026)
- [AI Evaluation Should Require Standardized Item-Level Data Releases](https://mlsanity.com/paper/2604.03244/) (2026)
- [EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Management](https://mlsanity.com/paper/2606.15532/) (2026)
- [E3RG: Building Explicit Emotion-driven Empathetic Response Generation System with Multimodal Large Language Model](https://mlsanity.com/paper/2508.12854/) (2025)

## Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have highlighted the need for robust, comprehensive, and challenging benchmarks. Yet, research on evaluating their Emotional Intelligence (EI) is considerably limited. Existing benchmarks have two major shortcomings: first, they mainly focus on emotion recognition, neglecting essential EI capabilities such as emotion regulation and thought facilitation through emotion understanding; second, they are primarily constructed from existing datasets, which include frequent patterns, explicit information, and annotation errors, leading to unreliable evaluation. We propose EmoBench, a benchmark that draws upon established psychological theories and proposes a comprehensive definition for machine EI, including Emotional Understanding and Emotional Application. EmoBench includes a set of 400 hand-crafted questions in English and Chinese, which are meticulously designed to require thorough reasoning and understanding. Our findings reveal a considerable gap between the EI of existing LLMs and the average human, highlighting a promising direction for future research. Our code and data are publicly available at https://github.com/Sahandfer/EmoBench.