Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models
Not on the map. Why?
Abstract (arXiv)
Self-supervised speech models (S3Ms) achieve strong downstream performance, yet their learned representations remain poorly understood under natural and adversarial perturbations. Prior studies rely on representation similarity or global dimensionality, offering limited visibility into local geometric changes. We ask: how do perturbations deform local geometry, and do these shifts track downstream automatic speech recognition (ASR) degradation? To address this, we present GRIDS, a framework using Local Intrinsic Dimensionality (LID) across layer-wise representations in WavLM and wav2vec 2.0. We find that LID increases for all low signal-to noise ratio (SNR) perturbations and diverges at high SNR: benign noise converges toward the clean profile, while adversarial inputs retain early-layer LID elevation. We show LID elevation co-occurs with increased WER, and that layer-wise LID features enable anomaly detection (AUROC 0.78-1.00), opening the door to transcript-free monitoring in S3Ms.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
26 of 45 references.
References · 26
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
- 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 315
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 282
- 2019Intrinsic dimension of data representations in deep neural networkscited by 282
- 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
- 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
14 more
- 2016On Deep Multi-View Representation Learning: Objectives and Optimizationcited by 162
- 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
- 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 153
- 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
- 2018On the Intrinsic Dimensionality of Image Representationscited by 57
- 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 46
- 2022A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognitioncited by 22
- 2025Detecting Backdoor Samples in Contrastive Language Image Pretrainingcited by 19
- 2022Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptationcited by 17
- 2021Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstructioncited by 13
- 2019A Robust Approach for Securing Audio Classification Against Adversarial Attackscited by 12
- 2024Towards Automatic Assessment of Self-Supervised Speech Models using Rankcited by 5
- 2025Towards Early Prediction of Self-Supervised Speech Model Performancecited by 5
- 2021Local Intrinsic Dimensionality Signals Adversarial Perturbationscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.