# Probing Layer-Wise Robustness and Sensitivity of Speech Enhancement Models

arXiv: https://arxiv.org/abs/2512.00482 · HTML: https://mlsanity.com/paper/2512.00482/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth](https://mlsanity.com/paper/2010.15327/) (2020)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra](https://mlsanity.com/paper/2305.13686/) (2023)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Similarity Analysis of Self-Supervised Speech Representations](https://mlsanity.com/paper/2010.11481/) (2020)
- [Demystifying TasNet: A Dissecting Approach](https://mlsanity.com/paper/1911.08895/) (2019)
- [A Systematic Comparison of Phonetic Aware Techniques for Speech Enhancement](https://mlsanity.com/paper/2206.11000/) (2022)
- [MUSE: Flexible Voiceprint Receptive Fields and Multi-Path Fusion Enhanced Taylor Transformer for U-Net-based Speech Enhancement](https://mlsanity.com/paper/2406.04589/) (2024)
- [RIR-Mega: a large-scale simulated room impulse response dataset for machine learning and room acoustics modeling](https://mlsanity.com/paper/2510.18917/) (2025)
- [Evaluation of Deep-Learning-Based Voice Activity Detectors and Room Impulse Response Models in Reverberant Environments](https://mlsanity.com/paper/2106.13511/) (2021)
- [Voice Activity Detection for Transient Noisy Environment Based on Diffusion Nets](https://mlsanity.com/paper/2106.13763/) (2021)
- [Objective Metrics to Evaluate Residual-Echo Suppression During Double-Talk](https://mlsanity.com/paper/2107.07471/) (2021)
- [MAPSS: Manifold-based Assessment of Perceptual Source Separation](https://mlsanity.com/paper/2509.09212/) (2025)

## Abstract (arXiv)

Speech enhancement (SE) models advance rapidly, yet how input degradation affects their internal representations remains underexplored. We introduce a probing framework to characterize how internal representations in SE models behave under controlled input degradation. We probe three SE models across controlled levels of signal-to-noise ratio (SNR) and reverberation, quantified by C50, measuring layer-wise similarity to clean references with Centered Kernel Alignment (CKA) and summarizing each layer by a linear fit against degradation level: the intercept measures robustness, whereas the slope measures sensitivity. All three models are sharply non-uniform across depth, but they organize that non-uniformity differently. MUSE and MP-SENet grow more sensitive with depth, the sharpest transitions falling at MUSE's skip-connection junctions, where encoder information is reintegrated; Demucs inverts the trend. A randomly initialized model shows a near-flat profile, with slopes one to two orders of magnitude smaller, and the profile forms during fine-tuning, indicating that it is induced by the enhancement objective rather than a particular model design. Because CKA saturates at the clean reference, intercept and slope are partly coupled; we derive the identity relating them and report a saturation spread statistic that indicates when their relationship is informative. Together, these results characterize where SE models are most sensitive to degradation. An exploratory analysis of whether residual variation tracks output-level quality, after controlling for SNR, shows that the speaker, rather than the utterance, must be treated as the sampling unit. Code and precomputed analysis artifacts for the main sweeps are publicly available.