# Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models

arXiv: https://arxiv.org/abs/2605.02715 · HTML: https://mlsanity.com/paper/2605.02715/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 26 of 26)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank](https://mlsanity.com/paper/2210.02885/) (2022)
- [Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training](https://mlsanity.com/paper/2104.01027/) (2021)
- [On the Intrinsic Dimensionality of Image Representations](https://mlsanity.com/paper/1803.09672/) (2018)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2201.08930/) (2022)
- [Detecting Backdoor Samples in Contrastive Language Image Pretraining](https://mlsanity.com/paper/2502.01385/) (2025)
- [Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptation](https://mlsanity.com/paper/2203.16104/) (2022)
- [Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction](https://mlsanity.com/paper/2110.15430/) (2021)
- [A Robust Approach for Securing Audio Classification Against Adversarial Attacks](https://mlsanity.com/paper/1904.10990/) (2019)
- [Towards Automatic Assessment of Self-Supervised Speech Models using Rank](https://mlsanity.com/paper/2409.10787/) (2024)
- [Towards Early Prediction of Self-Supervised Speech Model Performance](https://mlsanity.com/paper/2501.05966/) (2025)
- [Local Intrinsic Dimensionality Signals Adversarial Perturbations](https://mlsanity.com/paper/2109.11803/) (2021)

## Built on by (most cited first, 2 of 2)

- [Quieter Than the Room: Representation Drift and Task Robustness in Speech Encoders](https://mlsanity.com/paper/2609.27195/) (2026)
- [LAYERSCOPE: A Layerwise Characterization of Video and Multimodal Learned Representations](https://mlsanity.com/paper/2609.28086/) (2026)

## Abstract (arXiv)

Self-supervised speech models (S3Ms) achieve strong downstream performance, yet their learned representations remain poorly understood under natural and adversarial perturbations. Prior studies rely on representation similarity or global dimensionality, offering limited visibility into local geometric changes. We ask: how do perturbations deform local geometry, and do these shifts track downstream automatic speech recognition (ASR) degradation? To address this, we present GRIDS, a framework using Local Intrinsic Dimensionality (LID) across layer-wise representations in WavLM and wav2vec 2.0. We find that LID increases for all low signal-to noise ratio (SNR) perturbations and diverges at high SNR: benign noise converges toward the clean profile, while adversarial inputs retain early-layer LID elevation. We show LID elevation co-occurs with increased WER, and that layer-wise LID features enable anomaly detection (AUROC 0.78-1.00), opening the door to transcript-free monitoring in S3Ms.