# MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Models

arXiv: https://arxiv.org/abs/2511.00850 · HTML: https://mlsanity.com/paper/2511.00850/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 26 references that are in the map.
- First posted on arXiv 2 Nov 2025.

## Builds on (most cited first, 22 of 22)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024)
- [Kimi-Audio Technical Report](https://mlsanity.com/paper/2504.18425/) (2025)
- [AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension](https://mlsanity.com/paper/2402.07729/) (2024)
- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [Step-Audio 2 Technical Report](https://mlsanity.com/paper/2507.16632/) (2025)
- [PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support](https://mlsanity.com/paper/2106.01702/) (2021)
- [SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents](https://mlsanity.com/paper/2305.13040/) (2023)
- [SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words](https://mlsanity.com/paper/2406.13340/) (2024)
- [EmoBench: Evaluating the Emotional Intelligence of Large Language Models](https://mlsanity.com/paper/2402.12071/) (2024)
- [Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations](https://mlsanity.com/paper/2402.12786/) (2024)
- [Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation](https://mlsanity.com/paper/2406.07867/) (2024)
- [PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling](https://mlsanity.com/paper/2412.13660/) (2024)
- [Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models](https://mlsanity.com/paper/2412.05167/) (2024)
- [EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, Expressiveness, and Linguistic Challenges Using Model-as-a-Judge](https://mlsanity.com/paper/2505.23009/) (2025)
- [NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizations](https://mlsanity.com/paper/2508.04195/) (2025)
- [Does Your Voice Assistant Remember? Analyzing Conversational Context Recall and Utilization in Voice Interaction Models](https://mlsanity.com/paper/2502.19759/) (2025)
- [C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring Challenges in Complex Conversations](https://mlsanity.com/paper/2507.22968/) (2025)
- [SOVA-Bench: Benchmarking the Speech Conversation Ability for LLM-based Voice Assistant](https://mlsanity.com/paper/2506.02457/) (2025)
- [Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Model](https://mlsanity.com/paper/2506.08967/) (2025)

## Built on by (most cited first, 9 of 9)

- [The ICASSP 2026 HumDial Challenge: Benchmarking Human-like Spoken Dialogue Systems in the LLM Era](https://mlsanity.com/paper/2601.05564/) (2026)
- [Hear2Act: Benchmarking When Prosody Should Change What an Assistant Does](https://mlsanity.com/paper/2608.19515/) (2026)
- [HeartBench: Probing Core Dimensions of Anthropomorphic Intelligence in LLMs](https://mlsanity.com/paper/2512.21849/) (2025)
- [SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models](https://mlsanity.com/paper/2606.25990/) (2026)
- [Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging](https://mlsanity.com/paper/2607.10428/) (2026)
- [Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges](https://mlsanity.com/paper/2608.17605/) (2026)
- [Style Amnesia: Investigating Speaking Style Degradation and Mitigation in Multi-Turn Spoken Language Models](https://mlsanity.com/paper/2512.23578/) (2025)
- [GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human](https://mlsanity.com/paper/2605.28882/) (2026)
- [CallScreenBench: Benchmarking Small Language Models as Phone Secretaries](https://mlsanity.com/paper/2608.01033/) (2026)

## Abstract (arXiv)

Spoken Dialogue Models (SDMs) have advanced rapidly, yet their ability to sustain genuinely interactive multi-turn conversations remains underexplored, as most benchmarks focus on single-turn exchanges. We introduce Multi-Bench, the first benchmark explicitly designed to evaluate SDMs in multi-turn interactive dialogue with an emphasis on emotional intelligence. Multi-Bench employs a hierarchical structure with a basic track for emotion understanding and reasoning and an advanced track for emotion support and application. It comprises five carefully designed tasks and about 3.2K samples, ranging from emotion recognition to complex reasoning and interactive dialogue, supported by a reproducible evaluation framework. We evaluate six representative SDMs on eight subsets of Multi-Bench. Results show that while current SDMs achieve good performance on basic understanding tasks, they still have room for improvement in advanced multi-turn interactive dialogue and reasoning-related tasks, particularly in emotion awareness and application.