mlsanityData to 7 Oct 2026

MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models

arXiv 2603.09714 ↗First posted 10 Mar 2026cs.SD cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

While multi-audio understanding is critical for large audio-language models (LALMs), it remains underexplored. We introduce MUGEN, a comprehensive benchmark evaluating this capability across speech, general audio, and music. Our experiments reveal consistent weaknesses in multi-audio settings, and performance degrades sharply as the number of concurrent audio inputs increases, identifying input scaling as a fundamental bottleneck. We further investigate training-free strategies and observe that Audio-Permutational Self-Consistency, which diversifies the order of audio candidates, helps models form more robust aggregated predictions, yielding up to 6.28% accuracy gains. Combining this permutation strategy with Chain-of-Thought further improves performance to 6.74%. These results expose blind spots in current LALMs and provide a foundation for evaluating complex auditory comprehension.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 47 of its 59 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

47 of 59 references.

References · 47

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  10. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  11. 2025Qwen2.5-Omni Technical Reportcited by 704
  12. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
35 more
  1. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  2. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  3. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 215
  4. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
  5. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  6. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  7. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  8. 2026Qwen3-TTS Technical Reportcited by 106
  9. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 93
  10. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 70
  11. 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 67
  12. 2025MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligencecited by 59
  13. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  14. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 43
  15. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 42
  16. 2025Voxtralcited by 41
  17. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
  18. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 31
  19. 2024Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generationcited by 27
  20. 2025Mellow: a small audio language model for reasoningcited by 27
  21. 2025Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Modelscited by 27
  22. 2024Building a Taiwanese Mandarin Spoken Language Model: A First Attemptcited by 22
  23. 2024Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Modelscited by 20
  24. 2024Lyra: An Efficient and Speech-Centric Framework for Omni-Cognitioncited by 20
  25. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  26. 2025A Preliminary Exploration with GPT-4o Voice Modecited by 16
  27. 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognitioncited by 14
  28. 2023Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languagescited by 12
  29. 2025ADIFF: Explaining audio difference using natural languagecited by 8
  30. 2024DisfluencySpeech -- Single-Speaker Conversational Speech Dataset with Paralanguagecited by 6
  31. 2024Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Modelscited by 6
  32. 2023Exploring In-Context Learning of Textless Speech Language Model for Speech Classification Taskscited by 5
  33. 2025SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Modelscited by 5
  34. 2025Hearing the Order: Investigating Position Bias in Large Audio-Language Modelscited by 4
  35. 2025Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluationcited by 5
  2. 2026Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
  2. 2025Towards Reliable Large Audio Language Model
  3. 2026MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models
  4. 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
  5. 2026Continuous Audio Thinking for Large Audio Language Models
  6. 2026AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
  7. 2025Scaling Auditory Cognition via Test-Time Compute in Audio Language Models
  8. 2026EXAM2: Extending Audio Understanding in Multilingual and Multimodal Analysis