mlsanityData to 13 Oct 2026

EmoBench: Evaluating the Emotional Intelligence of Large Language Models

arXiv 2402.12071 ↗First posted 19 Feb 2024cs.CL cs.AIcited by 53

Not on the map. Why?

Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have highlighted the need for robust, comprehensive, and challenging benchmarks. Yet, research on evaluating their Emotional Intelligence (EI) is considerably limited. Existing benchmarks have two major shortcomings: first, they mainly focus on emotion recognition, neglecting essential EI capabilities such as emotion regulation and thought facilitation through emotion understanding; second, they are primarily constructed from existing datasets, which include frequent patterns, explicit information, and annotation errors, leading to unreliable evaluation. We propose EmoBench, a benchmark that draws upon established psychological theories and proposes a comprehensive definition for machine EI, including Emotional Understanding and Emotional Application. EmoBench includes a set of 400 hand-crafted questions in English and Chinese, which are meticulously designed to require thorough reasoning and understanding. Our findings reveal a considerable gap between the EI of existing LLMs and the average human, highlighting a promising direction for future research. Our code and data are publicly available at https://github.com/Sahandfer/EmoBench.

Key facts. Cited by 53 papers in the map (data through 13 Oct 2026). Builds on 23 of its 68 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

23 of 68 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  3. 2023Qwen Technical Reportcited by 3,310
  4. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,029
  5. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 705
  6. 2023Baichuan 2: Open Large-scale Language Modelscited by 668
  7. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  8. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 637
  9. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 481
  10. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 470
  11. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 357
  12. 2021Towards Emotional Support Dialog Systemscited by 296
11 more
  1. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 290
  2. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  3. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  4. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 145
  5. 2021Can Machines Learn Morality? The Delphi Experimentcited by 141
  6. 2020Recognizing Emotion Cause in Conversationscited by 66
  7. 2022CICERO: A Dataset for Contextualized Commonsense Inference in Dialoguescited by 38
  8. 2023A Wide Evaluation of ChatGPT on Affective Computing Taskscited by 30
  9. 2023ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mindcited by 28
  10. 2022Chatbots for Mental Health Support: Exploring the Impact of Emohaa on Reducing Mental Distress in Chinacited by 17
  11. 2023Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognitioncited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 7; 2025: 19; 2026: 27.

53 papers.

Sort:

By area:

  1. 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Modelscited by 32
  2. 2024Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspectivecited by 25
  3. 2024Human-like object concept representations emerge naturally in multimodal large language modelscited by 24
  4. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  5. 2025Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Modelscited by 21
  6. 2026Large Language Model Reasoning Failurescited by 21
  7. 2025MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Modelscited by 17
  8. 2025SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactionscited by 16
  9. 2024From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMscited by 13
  10. 2024A Survey on Human-Centric LLMscited by 13
  11. 2024Academically intelligent LLMs are not necessarily socially intelligentcited by 12
  12. 2026HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialoguecited by 12
38 more
  1. 2024MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysiscited by 10
  2. 2025Large Language Models for Subjective Language Understanding: A Surveycited by 9
  3. 2025MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Modelscited by 9
  4. 2026Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoningcited by 8
  5. 2025LIFELONG SOTOPIA: Evaluating Social Intelligence of Language Agents Over Lifelong Social Interactionscited by 7
  6. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  7. 2025Emergence of Hierarchical Emotion Organization in Large Language Modelscited by 6
  8. 2026Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMscited by 6
  9. 2026AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignmentcited by 6
  10. 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understandingcited by 5
  11. 2026Social-R1: Towards Human-like Social Reasoning in LLMscited by 5
  12. 2025MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?cited by 4
  13. 2025On the Adaptive Psychological Persuasion of Large Language Modelscited by 3
  14. 2025Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMscited by 3
  15. 2026MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialoguecited by 3
  16. 2026EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Managementcited by 3
  17. 2025E3RG: Building Explicit Emotion-driven Empathetic Response Generation System with Multimodal Large Language Modelcited by 2
  18. 2025A Latent Variable Framework for Scaling Laws in Large Language Modelscited by 2
  19. 2026AI Evaluation Should Require Standardized Item-Level Data Releasescited by 2
  20. 2024CharacterBench: Benchmarking Character Customization of Large Language Modelscited by 1
  21. 2025CAMEO: Collection of Multilingual Emotional Speech Corporacited by 1
  22. 2026Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathycited by 1
  23. 2026Emotion is Not Just a Label: Latent Emotional Factors in LLM Processingcited by 1
  24. 2026MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisalcited by 1
  25. 2026Quantitative Introspection in Language Models: Tracking Emotive States Across Conversationcited by 1
  26. 2026SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Modelscited by 1
  27. 2026ESC: Emotional Self-Correction for Reliable Vision-Language Modelscited by 1
  28. 2026MVEI & EmObserver: Empowering MLLM-Oriented Visual Emotional Intelligence via Emotion Statement Judgementcited by 1
  29. 2025EmoAssist: Emotional Assistant for Visual Impairment Communitycited by 0
  30. 2025Hybrid Emotion Recognition: Enhancing Customer Interactions Through Acoustic and Textual Analysiscited by 0
  31. 2025Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundariescited by 0
  32. 2025Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligencecited by 0
  33. 2026CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Modelscited by 0
  34. 2026CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classificationcited by 0
  35. 2026AttuneBench: A Conversation-Based Benchmark for LLM Emotional Intelligencecited by 0
  36. 2026Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?cited by 0
  37. 2026Chehre: An Emoji-Prompted Dataset to Explore Perceptual Flexibility in Video Language Modelscited by 0
  38. 2026Quantifying the Affective Gap: A Zero-Shot Evaluation of LLMs on Fine-Grained Emotion Taxonomiescited by 0

50 most cited of 53 All 53 (JSON)