# SEE: Signal Embedding Energy for Quantifying Noise Interference in Large Audio Language Models

arXiv: https://arxiv.org/abs/2601.07331 · HTML: https://mlsanity.com/paper/2601.07331/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Builds on (most cited first, 21 of 21)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension](https://mlsanity.com/paper/2402.07729/) (2024)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [TaskBench: Benchmarking Large Language Models for Task Automation](https://mlsanity.com/paper/2311.18760/) (2023)
- [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Deep Learning for Audio Signal Processing](https://mlsanity.com/paper/1905.00078/) (2019)
- [MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence](https://mlsanity.com/paper/2508.13992/) (2025)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study](https://mlsanity.com/paper/2505.19598/) (2025)
- [Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2509.21749/) (2025)
- [When Silence Matters: The Impact of Irrelevant Audio on Text Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2510.00626/) (2025)
- [ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models](https://mlsanity.com/paper/2510.23558/) (2025)
- [Explainable Disentanglement on Discrete Speech Representations for Noise-Robust ASR](https://mlsanity.com/paper/2510.25150/) (2025)

## Built on by (most cited first, 3 of 3)

- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)
- [EchoDistill: Robust Large Audio Language Models via Noisy-to-Clean Self-Distillation](https://mlsanity.com/paper/2605.23954/) (2026)
- [Long-Term Memory-Guided Enhancement for Target Perception in Audio-Language Models](https://mlsanity.com/paper/2609.36577/) (2026)

## Abstract (arXiv)

Large Audio Language Models (LALMs) have been widely applied in real-time scenarios, such as in-car assistants and online meeting comprehension. In practice, audio inputs are often corrupted by device and environmental noise, leading to performance degradation. However, existing LALM studies on noise lack quantitative analysis and rely mainly on intuition and empirical observation, thus failing to understand practical robustness. To address this issue, we introduce Signal Embedding Energy (SEE), a method for quantifying the impact of noise intensity on LALM inputs, enabling the differentiation of LALM robustness in real-world deployments. SEE introduces a perspective based on structured activation subspaces derived from the model's internal representations, which more accurately captures its perception of noise than raw audio features. Across experiments, SEE exhibits a strong correlation with LALM performance, achieving a correlation of 0.98. Surprisingly, traditional audio denoising methods are only marginally effective for LALMs, and, in some cases, even increase SEE and impair performance. This suggests a mismatch between speech-centric denoising objectives and the noise sensitivity of modern LALMs. Therefore, we propose a mitigation strategy derived from SEE to denoise LALM inputs, outperforming existing denoising methods. This paper introduces a novel metric for noise quantification in LALMs, providing guidance for robustness improvements in real-world deployments.