# Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2208.08509 · HTML: https://mlsanity.com/paper/2208.08509/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 17 Aug 2022.

## Builds on (most cited first, 14 of 14)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training](https://mlsanity.com/paper/2104.01027/) (2021)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [Multi-task self-supervised learning for Robust Speech Recognition](https://mlsanity.com/paper/2001.09239/) (2020)
- [Did you hear that? Adversarial Examples Against Automatic Speech Recognition](https://mlsanity.com/paper/1801.00554/) (2018)
- [Universal Adversarial Perturbations for Speech Recognition Systems](https://mlsanity.com/paper/1905.03828/) (2019)
- [Wav2vec-C: A Self-supervised Model for Speech Representation Learning](https://mlsanity.com/paper/2103.08393/) (2021)
- [A Curriculum Learning Method for Improved Noise Robustness in Automatic Speech Recognition](https://mlsanity.com/paper/1606.06864/) (2016)
- [Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition](https://mlsanity.com/paper/2110.04934/) (2021)
- [Recent improvements of ASR models in the face of adversarial attacks](https://mlsanity.com/paper/2203.16536/) (2022)
- [Analyzing the Robustness of Unsupervised Speech Recognition](https://mlsanity.com/paper/2110.03509/) (2021)

## Abstract (arXiv)

We investigate robustness properties of pre-trained neural models for automatic speech recognition. Real life data in machine learning is usually very noisy and almost never clean, which can be attributed to various factors depending on the domain, e.g. outliers, random noise and adversarial noise. Therefore, the models we develop for various tasks should be robust to such kinds of noisy data, which led to the thriving field of robust machine learning. We consider this important issue in the setting of automatic speech recognition. With the increasing popularity of pre-trained models, it's an important question to analyze and understand the robustness of such models to noise. In this work, we perform a robustness analysis of the pre-trained neural models wav2vec2, HuBERT and DistilHuBERT on the LibriSpeech and TIMIT datasets. We use different kinds of noising mechanisms and measure the model performances as quantified by the inference time and the standard Word Error Rate metric. We also do an in-depth layer-wise analysis of the wav2vec2 model when injecting noise in between layers, enabling us to predict at a high level what each layer learns. Finally for this model, we visualize the propagation of errors across the layers and compare how it behaves on clean versus noisy data. Our experiments conform the predictions of Pasad et al. [2021] and also raise interesting directions for future work.