# Benchmarking Humans and Machines on Complex Multilingual Speech Understanding Tasks

arXiv: https://arxiv.org/abs/2509.17965 · HTML: https://mlsanity.com/paper/2509.17965/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2025.

## Built on by (most cited first, 1 of 1)

- [The Eloquence submission for Task 2 of the Interspeech 2026 MLC-SLM challenge](https://mlsanity.com/paper/2609.11724/) (2026)

## Abstract (arXiv)

Auditory attention and selective phase-locking are central to human speech understanding in complex acoustic scenes and cocktail party settings, yet these capabilities in multilingual subjects remain poorly understood. While machine understanding of natural speech has advanced in recent years, questions persist about comprehension of overlapped and mixed-channel speech. We propose a systematic paradigm for studying humans and machines in speech question-answering tasks in multilingual settings with clean and mixed-channel speech. For human listeners, selective attention to a target speaker was significantly better in their native language (L1) than in their second language (L2). For machine listening, speech-based large language models (LLMs) match or exceed human performance in clean, single-speaker conditions but often struggle to selectively attend in two-speaker settings. These results reveal a key divergence: humans rely on attentional cues that are more streamlined in their native language, whereas LLMs default to parallel information extraction which exceed human skills.