# MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models

arXiv: https://arxiv.org/abs/2608.22236 · HTML: https://mlsanity.com/paper/2608.22236/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 72 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Builds on (most cited first, 30 of 34)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension](https://mlsanity.com/paper/2402.07729/) (2024)
- [FMA: A Dataset For Music Analysis](https://mlsanity.com/paper/1612.01840/) (2016)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities](https://mlsanity.com/paper/2406.11768/) (2024)
- [Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities](https://mlsanity.com/paper/2503.03983/) (2025)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.04779/) (2025)
- [MiMo-Audio: Audio Language Models are Few-Shot Learners](https://mlsanity.com/paper/2512.23808/) (2025)
- [Creating A Multi-track Classical Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications](https://mlsanity.com/paper/1612.08727/) (2016)
- [Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality](https://mlsanity.com/paper/2505.18227/) (2025)
- [Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration](https://mlsanity.com/paper/2505.20256/) (2025)
- [QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language](https://mlsanity.com/paper/2502.09723/) (2025)
- [Reasoning-Enhanced Domain-Adaptive Pretraining of Multimodal Large Language Models for Short Video Content Governance](https://mlsanity.com/paper/2509.21486/) (2025)
- [QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions](https://mlsanity.com/paper/2503.20290/) (2025)
- [Adaptive Information Control for Search-Augmented LLM Reasoning](https://mlsanity.com/paper/2602.01672/) (2026)
- [Rethinking Layer-Wise Information Allocation for Vision Foundation Model Adaptation](https://mlsanity.com/paper/2607.21973/) (2026)
- [AuTAgent: A Reinforcement Learning Framework for Tool-Augmented Audio Reasoning](https://mlsanity.com/paper/2602.13685/) (2026)
- [A Survey of Advancing Audio Super-Resolution and Bandwidth Extension from Discriminative to Generative Models](https://mlsanity.com/paper/2605.16681/) (2026)
- [Embedding-based Retrieval in Multimodal Content Moderation](https://mlsanity.com/paper/2507.01066/) (2025)
- [AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning](https://mlsanity.com/paper/2602.10439/) (2026)

## Built on by (most cited first, 2 of 2)

- [Decision-Oriented Uncertainty Quantification for Risk Control in Earth System Spatiotemporal Foundation Models](https://mlsanity.com/paper/2609.14821/) (2026)
- [CoRe: Coherence and Relational Alignment for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2609.19670/) (2026)

## Abstract (arXiv)

Large audio-language models (LALMs) have shown promising progress in understanding speech, music, and general sound events, yet their ability to reason about how audio signals are degraded remains underexplored. Existing benchmarks primarily evaluate semantic understanding, event recognition, or high-level audio reasoning, leaving a basic question unanswered: Do LALMs understand the differences in audio quality? We introduce MRMAD, a Multi-Round Multi-Audio Degradation benchmark for evaluating audio degradation perception and understanding in LALMs. MRMAD spans speech, music, and sound, and frames evaluation as multi-turn dialogues across multiple audio inputs, requiring models to identify types of degradation, compare severity, and perceive corruption changes across turns. Unlike current single-turn audio-language benchmarks, MRMAD evaluates whether LALMs can maintain consistent degradation hypotheses with new evidence and comprehend low-level acoustic phenomena over multi-turn dialogues. Through a systematic evaluation of 18 representative LALMs from non-thinking to reasoning and Omni models, we find that current models often recognize coarse content while failing to diagnose, compare, or reason about degradations reliably. Human evaluations further reveal a significant perception gap between LALMs and human listeners. MRMAD thus exposes a critical yet overlooked aspect of audio-language understanding and provides a diagnostic foundation for building future LALMs that are robust to real-world acoustic conditions.