# M3-DuplexBench: A Multi-Turn, Multilingual, Multidomain Benchmark for Full-Duplex Spoken Dialogue Models

arXiv: https://arxiv.org/abs/2607.29125 · HTML: https://mlsanity.com/paper/2607.29125/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM](https://mlsanity.com/paper/2411.00774/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models](https://mlsanity.com/paper/2602.06053/) (2026)
- [Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models](https://mlsanity.com/paper/2507.23159/) (2025)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examiner](https://mlsanity.com/paper/2510.07838/) (2025)
- [SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model](https://mlsanity.com/paper/2505.15670/) (2025)
- [MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Models](https://mlsanity.com/paper/2511.10262/) (2025)
- [FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2507.19040/) (2025)
- [DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications](https://mlsanity.com/paper/2409.19020/) (2024)
- [From Turn-Taking to Synchronous Dialogue: A Survey of Full-Duplex Spoken Language Models](https://mlsanity.com/paper/2509.14515/) (2025)
- [SoulX-Duplug: Plug-and-Play Streaming State Prediction Module for Realtime Full-Duplex Speech Conversation](https://mlsanity.com/paper/2603.14877/) (2026)
- [Towards a Japanese Full-duplex Spoken Dialogue System](https://mlsanity.com/paper/2506.02979/) (2025)
- [The ICASSP 2026 HumDial Challenge: Benchmarking Human-like Spoken Dialogue Systems in the LLM Era](https://mlsanity.com/paper/2601.05564/) (2026)

## Built on by (most cited first, 3 of 3)

- [Distributional Metrics for Evaluating Spoken Conversational Systems](https://mlsanity.com/paper/2609.31719/) (2026)
- [τ-Multilingual: Benchmarking Voice Agents Across Languages](https://mlsanity.com/paper/2609.35820/) (2026)
- [Conversation Is a Two-Body Problem: Dyadic Evaluation of Full-Duplex Dialogue Models](https://mlsanity.com/paper/2610.08125/) (2026)

## Abstract (arXiv)

Full-duplex spoken dialogue systems (FDSDSs) can listen while speaking, enabling natural conversational behaviors such as smooth turn-taking, backchannel handling, and user barge-in handling. However, controlled comparisons of FDSDSs in multi-turn interaction remain challenging, and existing benchmarks offer limited joint coverage of multiple languages, dialogue settings, interaction timing, and response content. We propose M3-DuplexBench, a multi-turn, multilingual, multidomain benchmark for controlled evaluation of FDSDSs. The benchmark jointly evaluates interaction timing and response content in English and Japanese across casual conversation and multi-turn question answering. The English and Japanese question-answering data are derived from a common source corpus, enabling cross-lingual comparisons of timing and content performance. We also provide a reference-conditioned multi-turn setting as an optional diagnostic for examining sensitivity to dialogue history and trajectory mismatch. Experiments with diverse FDSDSs reveal model-specific interaction patterns, differences across languages and domains, and model-dependent effects of dialogue context on both timing and content. The benchmark is available at https://github.com/nttcslab-sp/m3-duplexbench.